Commit Graph

112 Commits

Author SHA1 Message Date
Trisha Anand
9f63bfafd2
Adding JSON intelligent substitution for Rest Api plugin (#3474)
* POC : null value getting handled correctly

* Refactor code. Tested with and without quotes. This is working! Woohoo!

* Added support for mustache binding to include json objects and arrays

* Added test for smart substitution.

* Added setting to turn on/off smart substitutions in Rest API plugin

* Handling turning on smart json substitution

* Added error handling

* Added config to turn on smart substitution in JSON to make the tests pass now

* Adding a beta tag to the API setting.

* Spelling error resolved.

* Incorporated review comments.

* Catching JSON Parse exception in http call.
2021-03-11 14:10:00 +05:30
Sumit Kumar
b6961ce7a4
Add timeout failure for mongo plugin test datasource (#3431)
- Mongo plugin client driver does not return with exception upon first failure - instead it keeps retrying. Hence, adding timeout error to report failure before the client thread cancels due to delay in response.
- It seems that the mongdb connection string cannot be directly used for ping test, hence skipping it.
2021-03-10 11:11:14 +05:30
Trisha Anand
cd45a1dd74
Added support for Arrays in Prepared Statement in Postgres (#3421)
* Added support for Arrays in Prepared Statement

* Minor code cleanup

* Super minor code readability changes

* Incorporated review comment
2021-03-09 13:17:42 +05:30
Sumit Kumar
800d305dab
Gracefully handle the timeout overflow / out of expected range errors (#3411)
- Expect max value of timeout as 60000 ms.
- If value exceeds max value then add error message to the list of invalids. This list is returned to the client in response body.
- Detect integer overflow exception (Number format exception) and override the value to 60000 ms.
2021-03-05 17:09:29 +05:30
Trisha Anand
91e308665b
Requests preparation by plugins instead of ActionService (#3159)
* Added action request in S3 plugin.

* Added request data in Dynamo plugin

* Added request in Elastic Search.

* Request in Firestore.

* Request added in Mongo

* MsSQL request added.

* Added MySQL request.

* Added Postgres request.

* Added redis request.

* Redshift doen.

* Catching AppsmithPluginExceptions at the plugin level itself to ensure that the request gets passed on as part of the result.

* Fixed failing plugin test failures

* Fixed AmazonS3 test failures.

* WIP post analytics working

* WIP : Making the request data confirm to existing analytics request data format.

* Fixed the headers in the analytics body.

* Migrations S3, Postgres to the accepted format for request in analytics

* Updated S3 action to be the query

* Migration completed for all plugins for analytics.

* Removed the old analytics event. Now sending only the new one.

* Ensuring all data is captured in S3 plugin request in case of error.

* Minor editing of firestore to ensure that the errors get caught by the plugin itself.

* Fixed test cases in Amazon S3

* Incorporated review comments.
2021-03-02 10:58:46 +05:30
Trisha Anand
31b20da347
Adding support for NULL value parameters in Prepared Statement. (#3255) 2021-02-27 18:59:51 +05:30
Sumit Kumar
63ff3c73fa
Fix execute on page load for api on app copy (#3150)
* first draft

* added fix without TC

* cleanup

* Update app/server/appsmith-server/src/main/java/com/appsmith/server/services/NewActionServiceImpl.java

Co-authored-by: Trisha Anand <trisha@appsmith.com>

* Update app/server/appsmith-server/src/main/java/com/appsmith/server/services/NewActionServiceImpl.java

Co-authored-by: Trisha Anand <trisha@appsmith.com>

* Update app/server/appsmith-server/src/main/java/com/appsmith/server/services/NewActionServiceImpl.java

Co-authored-by: Trisha Anand <trisha@appsmith.com>

* - add TC
- minor code refactor

* fix condition

* update clone page test

Co-authored-by: Trisha Anand <trisha@appsmith.com>
2021-02-24 10:21:03 +05:30
Shrikant Sharat Kandula
68bbc4fb28
Support server-side pagination for Firestore (#3128)
* POC for Firestore server-side pagination

* Load pagination information values from action configuration

* Get execution information for Firestore plugin

* Working implementation of pagination for Firestore

* Add tests for next and previou page navigations

* Require ordering to be set when paginating

* Remove commented code

* Don't report error on bad configuration

* Error out on usupported operation

Co-authored-by: Trisha Anand <trisha@appsmith.com>

* Move constant indices to constant fields

* Use executeParameterized instead of execute

Co-authored-by: Trisha Anand <trisha@appsmith.com>
2021-02-24 10:20:08 +05:30
Trisha Anand
e5574c1945
Support Prepared Statements in Postgres (#2967)
* Pushing minor editor form changes to ensure that prepared statement could be turned off.

* Code refactor to do variable substitution in PluginExecutor instead of action service.

* WIP : Prepared Statement handling in psql plugin

* WIP Prepared Statements.

* Working version of prepared statements

* Quote trimming added for post preparing sql statements. Now the unprepared statements and prepared statements do not require edits.

* Fixed existing test cases failing.

* Code formatting.

* Super minor code cleanup.

* Added migration for the existing postgres actions.

* Fixed failing test cases in ActionServiceTest.

* Minor change in the text for turning on and off prepared statements in the postgres query pane.

* Added test cases for prepared statement.

* Some minor comments for code readability

* Moved Prepared Statement setting from Action Configuration to Plugin Specified Templates since this setting does not make sense for all the DB plugins.

* Added function level comments

* Update app/server/appsmith-interfaces/src/main/java/com/appsmith/external/helpers/SqlStringUtils.java

Co-authored-by: Arpit Mohan <mohanarpit@users.noreply.github.com>

* Update app/server/appsmith-interfaces/src/main/java/com/appsmith/external/helpers/SqlStringUtils.java

Co-authored-by: Arpit Mohan <mohanarpit@users.noreply.github.com>

* Incorporated review comments.

* Fixed compile time error.

Co-authored-by: Arpit Mohan <mohanarpit@users.noreply.github.com>
2021-02-18 18:33:27 +05:30
Piyush Mishra
74b013d909
Encode params toggle (#3006)
* Makes client side changes to support actionConfiguration.encodeParamsToggle setting.
* Add migration code.

Co-authored-by: Sumit Kumar <sumit@appsmith.com>
Co-authored-by: Arpit Mohan <mohanarpit@users.noreply.github.com>
2021-02-16 20:31:35 +05:30
Nidhi
afc329fd7b
Added grant type Authorization code to OAuth2 (#2992)
Co-authored-by: Piyush <piyush@codeitout.com>
2021-02-16 17:58:03 +05:30
Sumit Kumar
4df595417f
return url on file create and fix image/video file upload issues with amazons3 plugin (#3012)
1. Return url on file create.
2. Fix image/video upload/download.
3. Change error msg format to directly print error msg without adding any prefix sentence.
2021-02-12 16:01:26 +05:30
Nidhi
8823f9d407
Added client credentials grant type for OAuth2 (#2649)
* Added client credentials grant type for OAuth2

Co-authored-by: Piyush <piyush@codeitout.com>
2021-02-11 17:58:06 +05:30
Sumit Kumar
e9e30d1fa4
Enable toggle for params encoding (#2968)
Enable toggle for params encoding
1. Add toggle param to ActionConfiguration class.
2. Add TCs to test toggle on/off/null.
2021-02-11 11:02:29 +05:30
Shrikant Sharat Kandula
ed1c926c8a
Add datasource structure support for DynamoDB (#2919) 2021-02-10 13:28:35 +05:30
Nidhi
c0e44d0cd9
Added null check for error messages (#2922)
* Added null check for error messages

* Added a generic null check for future null pointers from other sources.
2021-02-09 15:00:22 +05:30
Sumit Kumar
e6b77a1299
Stop logging some AppsmithPluginException to Sentry (#2351)
1. Move package  appsmith-interfaces/src/main/java/com/appsmith/external/pluginExceptions-> appsmith-interfaces/src/main/java/com/appsmith/external/exceptions/pluginExceptions
2. Move enum AppsmithErrorAction from appsmith-server/src/main/java/com/appsmith/server/exceptions/AppsmithError.java -> appsmith-interfaces/src/main/java/com/appsmith/external/exceptions/AppsmithErrorAction.java, so that both plugin exceptions and server exceptions could use the same enum.
3. Log exception based on the error action defined for each exception.
2021-02-04 15:24:30 +05:30
Trisha Anand
9e0f36a6f2
Improve Server thrown Error Messages' language (#2705)
* Improve Error Messages

* Added naming convention expansion for action name in the error

* Fixed failing test case assertion.

* Minor formatting.
2021-01-25 18:50:32 +05:30
dependabot[bot]
e781ceb30a
Bump jackson-databind in /app/server/appsmith-interfaces (#2652)
Bumps [jackson-databind](https://github.com/FasterXML/jackson) from 2.9.10.5 to 2.9.10.7.
- [Release notes](https://github.com/FasterXML/jackson/releases)
- [Commits](https://github.com/FasterXML/jackson/commits)

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-01-21 10:48:01 +05:30
Nidhi
e6a0b00a25
Added Oauth2 functionality for REST APIs (#2509)
* Added Oauth2 functionality for REST APIs

* Encrypted response

* Missed file

* Review comments and tests

* Removed broken test
2021-01-15 16:41:13 +05:30
Shrikant Sharat Kandula
d5e0fb18d8
Add migration to set isEncrypted field where missing (#2332) 2020-12-23 15:35:45 +05:30
Shrikant Sharat Kandula
c7f330c4c7 Missing isEncrypted should mean a true value 2020-12-23 14:23:03 +05:30
Shrikant Sharat Kandula
cfcd7aa0bb
Fix Firestore datasources not encrypting secret key JSON when editing (#2296)
* Fix encrypted field not being ignored in JSON

* Don't mask error in decrypting

* Revert masking of error in decrypting
2020-12-21 17:43:15 +05:30
Nidhi
e3cf285d1e
Fixed encryption related bugs for empty/new datasource (#2287) 2020-12-21 10:41:57 +05:30
Nidhi
5a36d17f7a
Fixed encryption related bugs for empty/new datasource (#2272) 2020-12-18 16:13:01 +05:30
Nidhi
a00c144e85
Subclassing authentication (#2215)
* Sublcassing authentication

* Removed TODO

* Review changes
2020-12-17 21:38:52 +05:30
Trisha Anand
2062c60ca3
Moved all the scheduling of the plugin executions to bounded elastic instead of elastic. (#1931) 2020-11-26 17:50:43 +05:30
Shrikant Sharat Kandula
74cd362057
Firestore Integration (#1799)
* Adding the skeleton for Firestore integration

* Adding the datasource & query editor forms

Also adding the database changelog for the firestore plugin

Commenting out the firestore.close() connection because that causes issues with multiple Firestore tenants running in the same JVM.

* Adding the code for fetching the structure of collections from Firestore

* Use single document path field for Firestore

* Fix potential NPE when datasource destroy timeouts

* Work in progress on collection level ops for Firestore

* Get documents in a collection now works

* Add collection level querying support

* Mild refactoring

* Fix NPE when some fields are missing

* Hide clientJSON as a password field for Firestore

* Make collection level querying reactive

* Make reactive

* Validate before connecting

* Add tests for all supported methods in Firestore

* Fix forms for Firestore with hidden fields

* Hide limit and order by fields when not needed

* Restore log entry deleted by mistake

* Use S3 URL for Firestore/Firebase logo

* Add comments detailing why some code is commented

* Make parsing JSON reactive and fix subscribe calls

* Fix reactive scheduler

Co-authored-by: Arpit Mohan <arpit@appsmith.com>
2020-11-23 18:42:33 +05:30
Nidhi
c7efe55c9c
Fixed timeout error message, added new type of error for plugin timeout (#1519)
* Fixed timeout error message, added new type of error for plugin timeouts

* Switched error code and added test for a secondary StaleConnectionException
2020-11-04 12:56:52 +05:30
Prashant Chaubey
bf69c7d66b
Adding basic structure for plugin integrating with Redis (#1085)
Co-authored-by: nitesh261193 <nitesh261193@gmail.com>
2020-10-22 13:26:58 +05:30
dependabot[bot]
668c8e9c1c
Bump junit from 4.11 to 4.13.1 in /app/server/appsmith-interfaces (#1161)
Bumps [junit](https://github.com/junit-team/junit4) from 4.11 to 4.13.1.
- [Release notes](https://github.com/junit-team/junit4/releases)
- [Changelog](https://github.com/junit-team/junit4/blob/main/doc/ReleaseNotes4.11.md)
- [Commits](https://github.com/junit-team/junit4/compare/r4.11...r4.13.1)

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2020-10-13 14:33:50 +05:30
Shrikant Sharat Kandula
13ddcc0cc9
Upgrade pf4j & test containers and fix plugin tests (#1156) 2020-10-13 13:42:14 +05:30
Shrikant Sharat Kandula
4f3de3ab9f
Datasource structure support for MongoDB (#641)
* Added structure for MongoDB datasources

* Fixed tests for MongoDB template queries
2020-10-01 10:41:29 +05:30
Shrikant Sharat Kandula
75ba90928e
Add datasource structure caching and MySQL structure support (#553)
* Implement caching for datasource structure

* Avoid using final fields so spring-data can load objects

* Use type variable for getStructure method

* Initial version with base structure for MySQL plugin

* Add tests for MySQL plugin structure

* Add sorting for keys in Postgres structure as well

* Show "primary key" instead of just "primary"

* Refactor to reduce inline magic strings
2020-09-17 13:59:46 +05:30
Shrikant Sharat Kandula
6c7e45d010
Add a connection type variable for Plugin implementations (#531)
* Add a connection type variable for Plugin implementations

This type variable is intended to represent the type of the
connection object, if any, that the plugin will use. This will
help make the implementations more robust by leveraging Java's
type checking instead of rudimentary type casts over the
connection objects.

* Fix missing typevar usage
2020-09-14 20:36:47 +05:30
Shrikant Sharat Kandula
3f5930e52e
Add datasource structure support for DB plugins (#523)
* Base interface for getting datasource structure for databases

* Add keys and constraints to structure for Postgres datasources

* Use connection from datasource context for computing structure

* Refactor context retrying into a separate method

* Add base datastructures for templates in entity explorer

* Fix spring circular dependency

* Add test for postgres datasource structure

* Generate column names and sample values for INSERT query

* Add LIMIT clause to generated SELECT query

* Fix tests for generated SELECT query

* Minor refactoring
2020-09-14 18:59:11 +05:30
Shrikant Sharat Kandula
b511c3ada9
Change return type of plugin execution to be more specific (#247) 2020-08-10 14:41:32 +05:30
Shrikant Sharat Kandula
dfcabab4cc
Handle stale database connection from datasources (#151)
* Handle stale database connection from datasources

* Fix potential secondary case of stale connection error

* Fix Postgres to MySQL

* Move validity check timeout to a constant field

* Add test for recovery when stale connection error is thrown
2020-07-24 12:18:25 +05:30
Trisha Anand
fd0f23b9cc
Database credentials encryption in MongoDB (#80)
* Encrypting the password stored in AuthenticationDTO for every db.

* Adding comment to the properties file to denote that adding encryption salt and password are mandatory to the server coming up.

* Added the encryption salt and password to server.yml to allow the github actions to succeed.

* Adding database migration to encrypt the existing passwords for authentication object (used for storing db connection username/password)

Changes to the installation script install.sh:

1. Instead of overwriting the existing encryption password or salt, giving the user an option to conserve the previous encryption credentials to ensure that the developer users do not lose access to their database configurations (passwords).
2. Added another file for writing encryption credentials (encryption.env) to ensure that we dont delete the encryption password and salt by mistake.
2020-07-14 14:45:08 +05:30
Trisha Anand
8734067cad
Resolving the dependabot vulnerabilities. (#69) 2020-07-09 21:12:16 +05:30
Shrikant Kandula
3217daab14 Fix Mustache rendering to work on object fields directly 2020-06-25 09:58:20 +00:00
Trisha Anand
0603da29ee User test - Add request URL and http method to the execution result 2020-06-19 16:24:58 +00:00
Trisha Anand
73757c3425 Merge branch 'release' into feature/acl-spring-object
# Conflicts:
#	.gitignore
#	appsmith-plugins/mongoPlugin/plugin.properties
#	appsmith-plugins/postgresPlugin/plugin.properties
#	appsmith-plugins/rapidApiPlugin/plugin.properties
#	appsmith-plugins/restApiPlugin/plugin.properties
#	appsmith-plugins/restApiPlugin/src/main/java/com/external/plugins/RestApiPlugin.java
#	appsmith-server/src/main/java/com/appsmith/server/constants/FieldName.java
#	appsmith-server/src/main/java/com/appsmith/server/repositories/ActionRepository.java
#	appsmith-server/src/main/java/com/appsmith/server/services/ActionServiceImpl.java
#	appsmith-server/src/main/java/com/appsmith/server/services/DatasourceContextServiceImpl.java
#	appsmith-server/src/main/java/com/appsmith/server/services/DatasourceServiceImpl.java
#	appsmith-server/src/test/java/com/appsmith/server/services/ActionServiceTest.java
#	appsmith-server/src/test/java/com/appsmith/server/services/ApplicationServiceTest.java
#	appsmith-server/src/test/java/com/appsmith/server/services/LayoutServiceTest.java
#	appsmith-server/src/test/java/com/appsmith/server/services/PageServiceTest.java
#	build.sh
2020-06-12 19:14:31 +05:30
Arpit Mohan
bbd33c2968 Removing the query field in ActionConfiguration and using String body as the field to store the query
This is to simplify the query pane on the frontend client. The client doesn't need to have separate interfaces for sql and non-sql plugins. All queries will be sent to the server in the form of a String that is parsed in different formats based on the plugin.

Also adding test cases for PostgresPlugin. Used TestContainers to simulate the postgres db in Docker inside the Java test itself. Very useful.
2020-06-09 12:12:27 +00:00
Trisha Anand
6475be63d1 Baseline code added for cascading the org level permissions to applications, pages and eventually actions. 2020-05-27 14:36:57 +00:00
Arpit Mohan
fb9e9fbf4c Request body and header in Action Execution Response 2020-05-14 02:08:46 +00:00
Trisha Anand
5829a92998 Merge branch 'release' into feature/acl-spring-object
# Conflicts:
#	appsmith-server/src/main/java/com/appsmith/server/services/ActionServiceImpl.java
2020-05-13 23:37:31 +05:30
Shrikant Kandula
9c48f42990 Rename ActionExecutionResult.shouldCacheResponse -> isExecutionSuccess. 2020-05-12 18:25:49 +05:30
Trisha Anand
957116409d Resolved the review comments. Changed the structure of the DTO returned. Added the userPermissions as part of the base domain leading to this field always computed before returning any object to the user. 2020-05-07 10:49:48 +00:00
Trisha Anand
2d1fe9d8b4 Merge branch 'release' into feature/acl-spring-object
# Conflicts:
#	appsmith-interfaces/src/main/java/com/appsmith/external/models/SSHPrivateKey.java
#	appsmith-server/src/main/java/com/appsmith/server/filters/AclFilter.java
#	appsmith-server/src/main/java/com/appsmith/server/services/CurlImporterService.java
#	appsmith-server/src/test/java/com/appsmith/server/services/CurlImporterServiceTest.java
2020-04-28 18:13:06 +05:30