Commit Graph

124 Commits

Author SHA1 Message Date
Trisha Anand
554c45eef8
[Bug Fix] Prepared Statement -String starting with a number recognized as JSON object instead of String (#3710)
* String starting with float like 2.1 recognized as JSON object instead of String.

* Incorporated review comments.
2021-03-29 12:22:06 +05:30
Trisha Anand
b9fd72e89d
Smart Substitution Debugging - Adding Appsmith data type to each parameter in response's request object for execute (#3716)
* Smart Substitution Debugging - Adding appsmith data type to each parameter in response's request object for execute

* Incorporated review comments.

* Fixed failing test cases.

* Fixed MS-SQL test failures.
2021-03-26 23:29:12 +05:30
Nidhi
683176fc48
Increased default limit for webclient payload (#3718)
* Increased default limit for webclient payload
2021-03-26 21:34:10 +05:30
Sumit Kumar
e347b61e53
Return hint on localhost url (#3611)
- Return hint for localhost URL on the following events:
Test datasource
Save datasource
Update REST API Url
Create datasource from REST API
When page gets loaded for first time.
- Hint message returned: "You may not able to access your localhost if Appsmith is running inside a docker container or on the cloud. Please check out Appsmith's documentation to understand more."
- It has been decided as part of actions pod meeting that these messages will not be persisted.
2021-03-26 15:42:34 +05:30
Trisha Anand
c420d0207f
Supporting timestamp datatype in Prepared Statement (#3690)
* WIP : Adding timestamp datatype in Prepared Statement

* Added simple tests for commonly used data types' identification

* Added plugin level test cases for timestamp with Prepared Statement to Postgres
2021-03-26 13:19:22 +05:30
Trisha Anand
ed1a0b4045
Refactor the smart substitution code to extract common code at plugin interface level. (#3497) 2021-03-25 19:10:42 +05:30
Trisha Anand
c6e4f91ffb
[Bug fix] A lax search for presence of binding during save page to match client algorithm to reduce page save error (#3698)
* Lax mustache binding check added to match the client side check when client recognizes a field to have a dynamic binding. This would reduce/remove bad bindings from throwing a 400 during save page.

* Added a test to assert that update layout does not fail in case the binding is technically incorrect because part of the mustache's lie inside quotes. Since client has a lax way of finding a dynamic path, server also follows suite.
2021-03-25 16:00:45 +05:30
Sumit Kumar
56f22edbe8
Return hint message on identical columns (#3656)
- Return hint message if identical column names are found in SQL query for postgres, MySQL, mssql, redshift plugin.
- Add a PluginUtils class to hold general utility functions for plugins.
2021-03-24 08:22:49 +05:30
Shri
a18e569f7d
Fix data serialization in analytics (#3677)
* Serialize object fields before sending to analytics

* Push event when application is published
2021-03-23 17:13:02 +05:30
Sumit Kumar
38c5639542
Provide non client certificate based SSL support for Mysql, Mongo, Postgres plugins. (#3518)
- Provide non client certificate based SSL support for Mysql, Mongo, Postgres plugins.
- Added a new option default, apart from the the SSL mode types supported by the driver. Default means that go with whatever default configuration driver provides.
2021-03-19 15:03:56 +05:30
Shri
bb1d0059d3
Clear OAuth tokens for forked datasources (#3609)
* Clear OAuth tokens for forked datasources

* Fix datasource duplicate finder in light of oAuth tokens

* Fix potential NPE
2021-03-18 21:08:56 +05:30
Shri
e1fb1203b5
Application forking fixes and improvements (#3519)
* Clone only those datasources that are actually used

* Retry when cloning app causes a name class

* Search for an existing datasource before cloning

* Test deep matching of datasources

* Refactoring, since now the tests pass

* Allow forking of owning applications

* Don't limit datasource naming suffix number
2021-03-15 22:19:40 +05:30
Trisha Anand
9f63bfafd2
Adding JSON intelligent substitution for Rest Api plugin (#3474)
* POC : null value getting handled correctly

* Refactor code. Tested with and without quotes. This is working! Woohoo!

* Added support for mustache binding to include json objects and arrays

* Added test for smart substitution.

* Added setting to turn on/off smart substitutions in Rest API plugin

* Handling turning on smart json substitution

* Added error handling

* Added config to turn on smart substitution in JSON to make the tests pass now

* Adding a beta tag to the API setting.

* Spelling error resolved.

* Incorporated review comments.

* Catching JSON Parse exception in http call.
2021-03-11 14:10:00 +05:30
Sumit Kumar
b6961ce7a4
Add timeout failure for mongo plugin test datasource (#3431)
- Mongo plugin client driver does not return with exception upon first failure - instead it keeps retrying. Hence, adding timeout error to report failure before the client thread cancels due to delay in response.
- It seems that the mongdb connection string cannot be directly used for ping test, hence skipping it.
2021-03-10 11:11:14 +05:30
Trisha Anand
cd45a1dd74
Added support for Arrays in Prepared Statement in Postgres (#3421)
* Added support for Arrays in Prepared Statement

* Minor code cleanup

* Super minor code readability changes

* Incorporated review comment
2021-03-09 13:17:42 +05:30
Sumit Kumar
800d305dab
Gracefully handle the timeout overflow / out of expected range errors (#3411)
- Expect max value of timeout as 60000 ms.
- If value exceeds max value then add error message to the list of invalids. This list is returned to the client in response body.
- Detect integer overflow exception (Number format exception) and override the value to 60000 ms.
2021-03-05 17:09:29 +05:30
Trisha Anand
91e308665b
Requests preparation by plugins instead of ActionService (#3159)
* Added action request in S3 plugin.

* Added request data in Dynamo plugin

* Added request in Elastic Search.

* Request in Firestore.

* Request added in Mongo

* MsSQL request added.

* Added MySQL request.

* Added Postgres request.

* Added redis request.

* Redshift doen.

* Catching AppsmithPluginExceptions at the plugin level itself to ensure that the request gets passed on as part of the result.

* Fixed failing plugin test failures

* Fixed AmazonS3 test failures.

* WIP post analytics working

* WIP : Making the request data confirm to existing analytics request data format.

* Fixed the headers in the analytics body.

* Migrations S3, Postgres to the accepted format for request in analytics

* Updated S3 action to be the query

* Migration completed for all plugins for analytics.

* Removed the old analytics event. Now sending only the new one.

* Ensuring all data is captured in S3 plugin request in case of error.

* Minor editing of firestore to ensure that the errors get caught by the plugin itself.

* Fixed test cases in Amazon S3

* Incorporated review comments.
2021-03-02 10:58:46 +05:30
Trisha Anand
31b20da347
Adding support for NULL value parameters in Prepared Statement. (#3255) 2021-02-27 18:59:51 +05:30
Sumit Kumar
63ff3c73fa
Fix execute on page load for api on app copy (#3150)
* first draft

* added fix without TC

* cleanup

* Update app/server/appsmith-server/src/main/java/com/appsmith/server/services/NewActionServiceImpl.java

Co-authored-by: Trisha Anand <trisha@appsmith.com>

* Update app/server/appsmith-server/src/main/java/com/appsmith/server/services/NewActionServiceImpl.java

Co-authored-by: Trisha Anand <trisha@appsmith.com>

* Update app/server/appsmith-server/src/main/java/com/appsmith/server/services/NewActionServiceImpl.java

Co-authored-by: Trisha Anand <trisha@appsmith.com>

* - add TC
- minor code refactor

* fix condition

* update clone page test

Co-authored-by: Trisha Anand <trisha@appsmith.com>
2021-02-24 10:21:03 +05:30
Shrikant Sharat Kandula
68bbc4fb28
Support server-side pagination for Firestore (#3128)
* POC for Firestore server-side pagination

* Load pagination information values from action configuration

* Get execution information for Firestore plugin

* Working implementation of pagination for Firestore

* Add tests for next and previou page navigations

* Require ordering to be set when paginating

* Remove commented code

* Don't report error on bad configuration

* Error out on usupported operation

Co-authored-by: Trisha Anand <trisha@appsmith.com>

* Move constant indices to constant fields

* Use executeParameterized instead of execute

Co-authored-by: Trisha Anand <trisha@appsmith.com>
2021-02-24 10:20:08 +05:30
Trisha Anand
e5574c1945
Support Prepared Statements in Postgres (#2967)
* Pushing minor editor form changes to ensure that prepared statement could be turned off.

* Code refactor to do variable substitution in PluginExecutor instead of action service.

* WIP : Prepared Statement handling in psql plugin

* WIP Prepared Statements.

* Working version of prepared statements

* Quote trimming added for post preparing sql statements. Now the unprepared statements and prepared statements do not require edits.

* Fixed existing test cases failing.

* Code formatting.

* Super minor code cleanup.

* Added migration for the existing postgres actions.

* Fixed failing test cases in ActionServiceTest.

* Minor change in the text for turning on and off prepared statements in the postgres query pane.

* Added test cases for prepared statement.

* Some minor comments for code readability

* Moved Prepared Statement setting from Action Configuration to Plugin Specified Templates since this setting does not make sense for all the DB plugins.

* Added function level comments

* Update app/server/appsmith-interfaces/src/main/java/com/appsmith/external/helpers/SqlStringUtils.java

Co-authored-by: Arpit Mohan <mohanarpit@users.noreply.github.com>

* Update app/server/appsmith-interfaces/src/main/java/com/appsmith/external/helpers/SqlStringUtils.java

Co-authored-by: Arpit Mohan <mohanarpit@users.noreply.github.com>

* Incorporated review comments.

* Fixed compile time error.

Co-authored-by: Arpit Mohan <mohanarpit@users.noreply.github.com>
2021-02-18 18:33:27 +05:30
Piyush Mishra
74b013d909
Encode params toggle (#3006)
* Makes client side changes to support actionConfiguration.encodeParamsToggle setting.
* Add migration code.

Co-authored-by: Sumit Kumar <sumit@appsmith.com>
Co-authored-by: Arpit Mohan <mohanarpit@users.noreply.github.com>
2021-02-16 20:31:35 +05:30
Nidhi
afc329fd7b
Added grant type Authorization code to OAuth2 (#2992)
Co-authored-by: Piyush <piyush@codeitout.com>
2021-02-16 17:58:03 +05:30
Sumit Kumar
4df595417f
return url on file create and fix image/video file upload issues with amazons3 plugin (#3012)
1. Return url on file create.
2. Fix image/video upload/download.
3. Change error msg format to directly print error msg without adding any prefix sentence.
2021-02-12 16:01:26 +05:30
Nidhi
8823f9d407
Added client credentials grant type for OAuth2 (#2649)
* Added client credentials grant type for OAuth2

Co-authored-by: Piyush <piyush@codeitout.com>
2021-02-11 17:58:06 +05:30
Sumit Kumar
e9e30d1fa4
Enable toggle for params encoding (#2968)
Enable toggle for params encoding
1. Add toggle param to ActionConfiguration class.
2. Add TCs to test toggle on/off/null.
2021-02-11 11:02:29 +05:30
Shrikant Sharat Kandula
ed1c926c8a
Add datasource structure support for DynamoDB (#2919) 2021-02-10 13:28:35 +05:30
Nidhi
c0e44d0cd9
Added null check for error messages (#2922)
* Added null check for error messages

* Added a generic null check for future null pointers from other sources.
2021-02-09 15:00:22 +05:30
Sumit Kumar
e6b77a1299
Stop logging some AppsmithPluginException to Sentry (#2351)
1. Move package  appsmith-interfaces/src/main/java/com/appsmith/external/pluginExceptions-> appsmith-interfaces/src/main/java/com/appsmith/external/exceptions/pluginExceptions
2. Move enum AppsmithErrorAction from appsmith-server/src/main/java/com/appsmith/server/exceptions/AppsmithError.java -> appsmith-interfaces/src/main/java/com/appsmith/external/exceptions/AppsmithErrorAction.java, so that both plugin exceptions and server exceptions could use the same enum.
3. Log exception based on the error action defined for each exception.
2021-02-04 15:24:30 +05:30
Trisha Anand
9e0f36a6f2
Improve Server thrown Error Messages' language (#2705)
* Improve Error Messages

* Added naming convention expansion for action name in the error

* Fixed failing test case assertion.

* Minor formatting.
2021-01-25 18:50:32 +05:30
dependabot[bot]
e781ceb30a
Bump jackson-databind in /app/server/appsmith-interfaces (#2652)
Bumps [jackson-databind](https://github.com/FasterXML/jackson) from 2.9.10.5 to 2.9.10.7.
- [Release notes](https://github.com/FasterXML/jackson/releases)
- [Commits](https://github.com/FasterXML/jackson/commits)

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-01-21 10:48:01 +05:30
Nidhi
e6a0b00a25
Added Oauth2 functionality for REST APIs (#2509)
* Added Oauth2 functionality for REST APIs

* Encrypted response

* Missed file

* Review comments and tests

* Removed broken test
2021-01-15 16:41:13 +05:30
Shrikant Sharat Kandula
d5e0fb18d8
Add migration to set isEncrypted field where missing (#2332) 2020-12-23 15:35:45 +05:30
Shrikant Sharat Kandula
c7f330c4c7 Missing isEncrypted should mean a true value 2020-12-23 14:23:03 +05:30
Shrikant Sharat Kandula
cfcd7aa0bb
Fix Firestore datasources not encrypting secret key JSON when editing (#2296)
* Fix encrypted field not being ignored in JSON

* Don't mask error in decrypting

* Revert masking of error in decrypting
2020-12-21 17:43:15 +05:30
Nidhi
e3cf285d1e
Fixed encryption related bugs for empty/new datasource (#2287) 2020-12-21 10:41:57 +05:30
Nidhi
5a36d17f7a
Fixed encryption related bugs for empty/new datasource (#2272) 2020-12-18 16:13:01 +05:30
Nidhi
a00c144e85
Subclassing authentication (#2215)
* Sublcassing authentication

* Removed TODO

* Review changes
2020-12-17 21:38:52 +05:30
Trisha Anand
2062c60ca3
Moved all the scheduling of the plugin executions to bounded elastic instead of elastic. (#1931) 2020-11-26 17:50:43 +05:30
Shrikant Sharat Kandula
74cd362057
Firestore Integration (#1799)
* Adding the skeleton for Firestore integration

* Adding the datasource & query editor forms

Also adding the database changelog for the firestore plugin

Commenting out the firestore.close() connection because that causes issues with multiple Firestore tenants running in the same JVM.

* Adding the code for fetching the structure of collections from Firestore

* Use single document path field for Firestore

* Fix potential NPE when datasource destroy timeouts

* Work in progress on collection level ops for Firestore

* Get documents in a collection now works

* Add collection level querying support

* Mild refactoring

* Fix NPE when some fields are missing

* Hide clientJSON as a password field for Firestore

* Make collection level querying reactive

* Make reactive

* Validate before connecting

* Add tests for all supported methods in Firestore

* Fix forms for Firestore with hidden fields

* Hide limit and order by fields when not needed

* Restore log entry deleted by mistake

* Use S3 URL for Firestore/Firebase logo

* Add comments detailing why some code is commented

* Make parsing JSON reactive and fix subscribe calls

* Fix reactive scheduler

Co-authored-by: Arpit Mohan <arpit@appsmith.com>
2020-11-23 18:42:33 +05:30
Nidhi
c7efe55c9c
Fixed timeout error message, added new type of error for plugin timeout (#1519)
* Fixed timeout error message, added new type of error for plugin timeouts

* Switched error code and added test for a secondary StaleConnectionException
2020-11-04 12:56:52 +05:30
Prashant Chaubey
bf69c7d66b
Adding basic structure for plugin integrating with Redis (#1085)
Co-authored-by: nitesh261193 <nitesh261193@gmail.com>
2020-10-22 13:26:58 +05:30
dependabot[bot]
668c8e9c1c
Bump junit from 4.11 to 4.13.1 in /app/server/appsmith-interfaces (#1161)
Bumps [junit](https://github.com/junit-team/junit4) from 4.11 to 4.13.1.
- [Release notes](https://github.com/junit-team/junit4/releases)
- [Changelog](https://github.com/junit-team/junit4/blob/main/doc/ReleaseNotes4.11.md)
- [Commits](https://github.com/junit-team/junit4/compare/r4.11...r4.13.1)

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2020-10-13 14:33:50 +05:30
Shrikant Sharat Kandula
13ddcc0cc9
Upgrade pf4j & test containers and fix plugin tests (#1156) 2020-10-13 13:42:14 +05:30
Shrikant Sharat Kandula
4f3de3ab9f
Datasource structure support for MongoDB (#641)
* Added structure for MongoDB datasources

* Fixed tests for MongoDB template queries
2020-10-01 10:41:29 +05:30
Shrikant Sharat Kandula
75ba90928e
Add datasource structure caching and MySQL structure support (#553)
* Implement caching for datasource structure

* Avoid using final fields so spring-data can load objects

* Use type variable for getStructure method

* Initial version with base structure for MySQL plugin

* Add tests for MySQL plugin structure

* Add sorting for keys in Postgres structure as well

* Show "primary key" instead of just "primary"

* Refactor to reduce inline magic strings
2020-09-17 13:59:46 +05:30
Shrikant Sharat Kandula
6c7e45d010
Add a connection type variable for Plugin implementations (#531)
* Add a connection type variable for Plugin implementations

This type variable is intended to represent the type of the
connection object, if any, that the plugin will use. This will
help make the implementations more robust by leveraging Java's
type checking instead of rudimentary type casts over the
connection objects.

* Fix missing typevar usage
2020-09-14 20:36:47 +05:30
Shrikant Sharat Kandula
3f5930e52e
Add datasource structure support for DB plugins (#523)
* Base interface for getting datasource structure for databases

* Add keys and constraints to structure for Postgres datasources

* Use connection from datasource context for computing structure

* Refactor context retrying into a separate method

* Add base datastructures for templates in entity explorer

* Fix spring circular dependency

* Add test for postgres datasource structure

* Generate column names and sample values for INSERT query

* Add LIMIT clause to generated SELECT query

* Fix tests for generated SELECT query

* Minor refactoring
2020-09-14 18:59:11 +05:30
Shrikant Sharat Kandula
b511c3ada9
Change return type of plugin execution to be more specific (#247) 2020-08-10 14:41:32 +05:30
Shrikant Sharat Kandula
dfcabab4cc
Handle stale database connection from datasources (#151)
* Handle stale database connection from datasources

* Fix potential secondary case of stale connection error

* Fix Postgres to MySQL

* Move validity check timeout to a constant field

* Add test for recovery when stale connection error is thrown
2020-07-24 12:18:25 +05:30