Commit Graph

181 Commits

Author SHA1 Message Date
Nidhi
e6a0b00a25
Added Oauth2 functionality for REST APIs (#2509)
* Added Oauth2 functionality for REST APIs

* Encrypted response

* Missed file

* Review comments and tests

* Removed broken test
2021-01-15 16:41:13 +05:30
Sumit Kumar
4748014cee
Redshift plugin (#2440)
* introduce redshift plugin
2021-01-06 17:51:07 +05:30
Nidhi
de5003a98e
Added type parameter for REST API plugin, and changes for no auth (#2438)
* Added type parameter for REST API plugin, and changes for no auth

* Removed logs

* Converted to reactive
2021-01-05 17:59:50 +05:30
Trisha Anand
d5074ee7f7
Increased logs to debug future connection leaks. (#2437)
* Increased logs to debug future connection leaks.

* Fetch a connection from the pool only if a query exists.

* Minor comment added.

* Minor rewrite

* Code formatting.

* Update app/server/appsmith-plugins/postgresPlugin/src/main/java/com/external/plugins/PostgresPlugin.java

Co-authored-by: Shrikant Sharat Kandula <shrikant@appsmith.com>

* Added hikari cp pool stats to get database structure function as well.

Co-authored-by: Shrikant Sharat Kandula <shrikant@appsmith.com>
2021-01-05 15:34:55 +05:30
Trisha Anand
214f911060 Fixed merge conflict related build failure. 2021-01-04 11:24:00 +05:30
Trisha Anand
24ef325467
Merge branch 'master' into release 2021-01-04 11:18:13 +05:30
Trisha Anand
a3378397d7 Increasing pool size to 20 to reduce the number of times datasource becomes unusable. Symptomatic relief till I can solve for this at leisure over the next few days.
Note : Update/Insert are leading to connection leak
2021-01-01 01:24:26 +05:30
Trisha Anand
a70b95be37 Adding leak detection to hikari pool. Would be tracking this on logdna to ensure that we are catching any and all leaks that may be occuring with hikari. (#2374) 2020-12-28 18:29:50 +05:30
Trisha Anand
051d3c6c7f
Adding leak detection to hikari pool. Would be tracking this on logdna to ensure that we are catching any and all leaks that may be occuring with hikari. (#2374) 2020-12-28 18:26:33 +05:30
Sumit Kumar
6299c86f39
Add TC to test mongodb connection establishment (#2364)
1. Test that when a query is attempted to run on mongodb server but refused because of lack of authorization, then
also, it indicates a successful connection establishment.
2020-12-28 11:58:17 +05:30
Sumit Kumar
53a6160139
Revert "Redshift plugin (#2112)" (#2314)
This reverts commit 708ca7fedf.
2020-12-22 16:59:18 +05:30
Sumit Kumar
708ca7fedf
Redshift plugin (#2112)
1. Add new plugin to connect to AWS Redshift cluster and run query.
2. Add unit test cases.
2020-12-22 15:35:25 +05:30
Shrikant Sharat Kandula
ea6e897dde
Firestore ADD_TO_COLLECTION command support (#2305)
* Support ADD_TO_COLLECTION operation on Firestore

* Add test for Firestore add-to-collection command
2020-12-22 12:32:14 +05:30
Shrikant Sharat Kandula
fdc4f7b200
Fix invalid templates and API breaking on invalid templates (#2286) 2020-12-19 13:00:24 +05:30
Shrikant Sharat Kandula
5e04ca5c9d
Add ability to configure order of templates of plugins (#2279) 2020-12-18 20:45:08 +05:30
Nidhi
a00c144e85
Subclassing authentication (#2215)
* Sublcassing authentication

* Removed TODO

* Review changes
2020-12-17 21:38:52 +05:30
Trisha Anand
29706f7852
Removing printing the result for firestore plugin (#2191) 2020-12-14 14:48:50 +05:30
Trisha Anand
e5c44168f3
Configured pool size for hikariCP when new pool comes uo. (#2170) 2020-12-14 11:07:53 +05:30
Trisha Anand
74342f8598
Adding hikariCP for postgres (#2130) 2020-12-10 21:08:02 +05:30
Trisha Anand
746b0ab58e Changed log to sout in postgres plugin 2020-12-09 19:16:54 +05:30
Trisha Anand
888d79b06d Removing printing of result objects from plugin execution. 2020-12-09 19:03:51 +05:30
Trisha Anand
8283396056
Moved from boundedElastic to elastic to ensure that the server services the plugin execution request as soon as the request arrives. (#2090) 2020-12-08 22:04:14 +05:30
Sumit Kumar
db3636f7b0
mysql plugin - check if connection is stale before running query (#2080)
1. Check if mysql plugin connection is stale. If so, then return stale connection exception which triggers a retry after 
    establishing a new connection.
2. Refactor execute method code to remove code duplication.
3. Add a new test case to check if stale connection gets detected.
2020-12-08 21:03:31 +05:30
Sumit Kumar
b6abb8b067
Encode request body in Rest API plugin (#1988)
1. Encode request body in REST API plugin when content header for URL encode is set.
2. Add a test case to test the encoding function.
3. Add a test case to check non-encoding (raw) function.
2020-12-04 11:39:15 +05:30
Shrikant Sharat Kandula
9bbbf37ae0
Fix #1597: Add a signed issuer token to proxied requests, if configured (#1953) 2020-12-02 18:20:57 +05:30
Sumit Kumar
d347448b24
Propagte mysql test datasource failure to UI. (#2001)
1. Propagte mysql test datasource failure exception message to UI.
2. Change build TC to reflect the same.
2020-12-02 17:53:38 +05:30
Sumit Kumar
b3c3102c72
Add tests for mysql plugin to test all regularly used data types in mysql. (#1994) 2020-12-02 11:49:41 +05:30
Sumit Kumar
883fa44896
Make sql plugin use R2DBC driver instead of JDBC driver (#1936)
1. Add R2BC driver and remove JDBC driver usage in mysql plugin to make the plugin reactive.
2. Update dependencies in POM file.
3. Update mysql plugin to return result based on the type of sql query i.e select vs other queries. In case the queries are chained, then the last query is used for deciding the type.
4. Added a new TC for testing datasource and removed a non functional TC.
2020-11-28 18:04:56 +05:30
Trisha Anand
2062c60ca3
Moved all the scheduling of the plugin executions to bounded elastic instead of elastic. (#1931) 2020-11-26 17:50:43 +05:30
Arpit Mohan
36af9d5c5e Correcting the placeholder in firestore form.json 2020-11-24 11:00:27 +05:30
Shrikant Sharat Kandula
74cd362057
Firestore Integration (#1799)
* Adding the skeleton for Firestore integration

* Adding the datasource & query editor forms

Also adding the database changelog for the firestore plugin

Commenting out the firestore.close() connection because that causes issues with multiple Firestore tenants running in the same JVM.

* Adding the code for fetching the structure of collections from Firestore

* Use single document path field for Firestore

* Fix potential NPE when datasource destroy timeouts

* Work in progress on collection level ops for Firestore

* Get documents in a collection now works

* Add collection level querying support

* Mild refactoring

* Fix NPE when some fields are missing

* Hide clientJSON as a password field for Firestore

* Make collection level querying reactive

* Make reactive

* Validate before connecting

* Add tests for all supported methods in Firestore

* Fix forms for Firestore with hidden fields

* Hide limit and order by fields when not needed

* Restore log entry deleted by mistake

* Use S3 URL for Firestore/Firebase logo

* Add comments detailing why some code is commented

* Make parsing JSON reactive and fix subscribe calls

* Fix reactive scheduler

Co-authored-by: Arpit Mohan <arpit@appsmith.com>
2020-11-23 18:42:33 +05:30
Trisha Anand
c07264f1c1
Making Redis plugin blocking statements run on elastic thread pool (#1850) 2020-11-23 17:03:04 +05:30
Trisha Anand
772884e1a3
Making dynamo db's blocking commands run on elastic thread pool. (#1853) 2020-11-23 17:02:41 +05:30
Trisha Anand
d8cba1bf8c
Making MySql blocking commands run on elastic thread pool (#1862) 2020-11-23 17:02:18 +05:30
Trisha Anand
0bb99dd087
Elastic search plugin's blocking calls moved to elastic scheduler thread (#1855) 2020-11-23 17:01:34 +05:30
Trisha Anand
242afd5753
Making MS-SQL plugin blocking statements run on elastic thread pool (#1827)
* MS-SQL Plugin now creates connection and executes statements over elastic thread pool.

* Added a test to ensure that throwables are clean.
2020-11-23 11:28:01 +05:30
Trisha Anand
dc7a375543
Making postgres plugin blocking statements run on elastic threadpool. (#1821)
* Making postgres plugin blocking statements run on elastic threadpool.

* Moved the blocking calls to check for the connection validity, etc. also inside the Mono.fromCallable
2020-11-23 11:27:52 +05:30
Sumit Kumar
2c6d1bb361
Fix mongodb datasource test. (#1783)
1. Fix mongodb datasource cypress test failure.
2. Against certain mongodb instances, we expect to receive an exception of type "unauthorized" when testing datasource. Earlier, this case was caught and whitelisted until the recent change to replace mongodb infra with reactive mongodb infra.
2020-11-18 16:55:39 +05:30
Shrikant Sharat Kandula
2ca15ba57d
Fix #1758: columns duplicated in Postgres structure (#1767) 2020-11-18 11:19:55 +05:30
Sumit Kumar
2c2aa06e32
Add support for reactive streams with mongodb. (#1720)
Add support for reactive streams with mongodb. (fixes 1480)

1. Replace mongodb driver with reactive mongodb driver. Change APIs accordingly.
2. Use webflux + reactor framework to execute mongodb queries in event loop model.
3. Add test to test MongoPluginExecutor class' method "testDatasource".
2020-11-18 10:58:50 +05:30
Arpit Mohan
470f2fafcd
Adding support for https endpoints for ES plugin (#1445)
* Adding host:port validation checks to ES plugin & Redis plugin

Also correcting the assertions in the RedisPluginTest so that the error message on test failure is accurate.

* Removing the endpoint validation from datasourceServiceImpl

Moving the endpoint validation to the plugin implementation classes because there are databases that require complete HTTP URL in their configuration. Checking for http/https at the platform level affects the UX for the user & plugin developer when using a new integration. Hence, all plugins must implement their own client/server validations in their own implementations

* Adding tests to assert hostname validation in mysql & postgres plugins
2020-10-28 19:06:10 +05:30
Arpit Mohan
c6d4902e3d
Adding microsoft SQL server plugin integration (#1374) 2020-10-23 20:36:45 +05:30
akash-codemonk
9d27315213
Update datasource form config to include encrypted property (#1360) 2020-10-23 13:31:46 +05:30
Arpit Mohan
74fe08f8bf
Adding the database changelog for Redis (#1346)
1. Fixing the build by excluding the slf4j-api from redis-plugin pom.xml
2. Adding the editor.json and form.json for the query pane & datasource pane.
3. Adding array handling in the Redis response by feeding all the output into a "result" key
2020-10-22 16:40:19 +05:30
Prashant Chaubey
bf69c7d66b
Adding basic structure for plugin integrating with Redis (#1085)
Co-authored-by: nitesh261193 <nitesh261193@gmail.com>
2020-10-22 13:26:58 +05:30
dependabot[bot]
bc14b6ce4b
Bump junit in /app/server/appsmith-plugins/elasticSearchPlugin (#1331)
Bumps [junit](https://github.com/junit-team/junit4) from 4.11 to 4.13.1.
- [Release notes](https://github.com/junit-team/junit4/releases)
- [Changelog](https://github.com/junit-team/junit4/blob/main/doc/ReleaseNotes4.11.md)
- [Commits](https://github.com/junit-team/junit4/compare/r4.11...r4.13.1)

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2020-10-21 22:20:49 +05:30
dependabot[bot]
a4872f3cb6
Bump junit in /app/server/appsmith-plugins/dynamoPlugin (#1335)
Bumps [junit](https://github.com/junit-team/junit4) from 4.11 to 4.13.1.
- [Release notes](https://github.com/junit-team/junit4/releases)
- [Changelog](https://github.com/junit-team/junit4/blob/main/doc/ReleaseNotes4.11.md)
- [Commits](https://github.com/junit-team/junit4/compare/r4.11...r4.13.1)

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2020-10-21 17:57:44 +05:30
Shrikant Sharat Kandula
2a53abd8a6
Add DynamoDB Plugin (#1167)
* Added DynamoPlugin to integrate with DynamoDB

* Connection management for DynamoPlugin

* Process action from a single JSON object as input

* Moved ListTables command to separate class

* Add missing UI definition files

* Use reflection to build request objects for AWS SDK

* All DynamoDB actions are now run purely by reflection

* Serialize responses to JSON

* Remove some unused/commented code and some refactorings

* Add non-working test with generic dynamodb container

* Added working test for ListTables action

* Added test for PutItem action

* Add tests for get and update actions

* Added comments and some refactoring

* Removed unused Command class.

* Added more tests for map to SDK object conversion

* Add templates

* Add option to set an endpoint override

* Fix Dynamo plugin tests when ~/.aws is missing

* Add documentation link for DynamoDB plugin

* Fix validation checks

* Remove debug log

* Added comments on expected query structure

* Fix incorrect error condition with incorrect request format

* Add comment for converting action to method name

* Error out if region is missing

* Remove unused throws declaration

* Use rich form for action configuration

* Removed templates

* Use PNG for logo images

* Use PNG logos for all plugins

* Avoid hard-coded field names

* Change logo to PNG only for ElasticSearch for now

* Wrap errors in AppsmithPluginException

* Typo

Co-authored-by: Trisha Anand <trisha@appsmith.com>

* Fix changeset order number

Co-authored-by: Trisha Anand <trisha@appsmith.com>

* Minor refactoring and fixed error messaging

Co-authored-by: Trisha Anand <trisha@appsmith.com>
2020-10-21 17:45:54 +05:30
Shrikant Sharat Kandula
fa1a0549ff
Add ElasticSearch integration (#1181)
* add elasticSearchPlugin

* Fix container startup in tests

* Add elasticsearch dependency

* Get plugin to a base working state

* Add templates and tests for all Document APIs

* Add support for bulk queries

* Add test and template for bulk operations

* Use rich form for action configuration

* Add test API for ElasticSearch

* Use rich form's values for plugin execution

* Add authorization header support

* Fix tests after config object use changes

* Add test for bulk requests with nd-json body

* Remove templates and minor refactoring

* Fix potential NPE with null body

Co-authored-by: Trisha Anand <trisha@appsmith.com>

* Add datasource validation for endpoint

* Wrap errors in AppsmithPluginException

Co-authored-by: Suman Patra <spatra@akamai.com>
Co-authored-by: Trisha Anand <trisha@appsmith.com>
2020-10-21 15:34:29 +05:30
Arpit Mohan
2bb093a5d7 Removing Mssql plugin 2020-10-20 22:27:29 +05:30