1. Check if mysql plugin connection is stale. If so, then return stale connection exception which triggers a retry after
establishing a new connection.
2. Refactor execute method code to remove code duplication.
3. Add a new test case to check if stale connection gets detected.
1. Encode request body in REST API plugin when content header for URL encode is set.
2. Add a test case to test the encoding function.
3. Add a test case to check non-encoding (raw) function.
1. Add R2BC driver and remove JDBC driver usage in mysql plugin to make the plugin reactive.
2. Update dependencies in POM file.
3. Update mysql plugin to return result based on the type of sql query i.e select vs other queries. In case the queries are chained, then the last query is used for deciding the type.
4. Added a new TC for testing datasource and removed a non functional TC.
* Adding the skeleton for Firestore integration
* Adding the datasource & query editor forms
Also adding the database changelog for the firestore plugin
Commenting out the firestore.close() connection because that causes issues with multiple Firestore tenants running in the same JVM.
* Adding the code for fetching the structure of collections from Firestore
* Use single document path field for Firestore
* Fix potential NPE when datasource destroy timeouts
* Work in progress on collection level ops for Firestore
* Get documents in a collection now works
* Add collection level querying support
* Mild refactoring
* Fix NPE when some fields are missing
* Hide clientJSON as a password field for Firestore
* Make collection level querying reactive
* Make reactive
* Validate before connecting
* Add tests for all supported methods in Firestore
* Fix forms for Firestore with hidden fields
* Hide limit and order by fields when not needed
* Restore log entry deleted by mistake
* Use S3 URL for Firestore/Firebase logo
* Add comments detailing why some code is commented
* Make parsing JSON reactive and fix subscribe calls
* Fix reactive scheduler
Co-authored-by: Arpit Mohan <arpit@appsmith.com>
* Making postgres plugin blocking statements run on elastic threadpool.
* Moved the blocking calls to check for the connection validity, etc. also inside the Mono.fromCallable
1. Fix mongodb datasource cypress test failure.
2. Against certain mongodb instances, we expect to receive an exception of type "unauthorized" when testing datasource. Earlier, this case was caught and whitelisted until the recent change to replace mongodb infra with reactive mongodb infra.
Add support for reactive streams with mongodb. (fixes 1480)
1. Replace mongodb driver with reactive mongodb driver. Change APIs accordingly.
2. Use webflux + reactor framework to execute mongodb queries in event loop model.
3. Add test to test MongoPluginExecutor class' method "testDatasource".
* Adding host:port validation checks to ES plugin & Redis plugin
Also correcting the assertions in the RedisPluginTest so that the error message on test failure is accurate.
* Removing the endpoint validation from datasourceServiceImpl
Moving the endpoint validation to the plugin implementation classes because there are databases that require complete HTTP URL in their configuration. Checking for http/https at the platform level affects the UX for the user & plugin developer when using a new integration. Hence, all plugins must implement their own client/server validations in their own implementations
* Adding tests to assert hostname validation in mysql & postgres plugins
1. Fixing the build by excluding the slf4j-api from redis-plugin pom.xml
2. Adding the editor.json and form.json for the query pane & datasource pane.
3. Adding array handling in the Redis response by feeding all the output into a "result" key
* Added DynamoPlugin to integrate with DynamoDB
* Connection management for DynamoPlugin
* Process action from a single JSON object as input
* Moved ListTables command to separate class
* Add missing UI definition files
* Use reflection to build request objects for AWS SDK
* All DynamoDB actions are now run purely by reflection
* Serialize responses to JSON
* Remove some unused/commented code and some refactorings
* Add non-working test with generic dynamodb container
* Added working test for ListTables action
* Added test for PutItem action
* Add tests for get and update actions
* Added comments and some refactoring
* Removed unused Command class.
* Added more tests for map to SDK object conversion
* Add templates
* Add option to set an endpoint override
* Fix Dynamo plugin tests when ~/.aws is missing
* Add documentation link for DynamoDB plugin
* Fix validation checks
* Remove debug log
* Added comments on expected query structure
* Fix incorrect error condition with incorrect request format
* Add comment for converting action to method name
* Error out if region is missing
* Remove unused throws declaration
* Use rich form for action configuration
* Removed templates
* Use PNG for logo images
* Use PNG logos for all plugins
* Avoid hard-coded field names
* Change logo to PNG only for ElasticSearch for now
* Wrap errors in AppsmithPluginException
* Typo
Co-authored-by: Trisha Anand <trisha@appsmith.com>
* Fix changeset order number
Co-authored-by: Trisha Anand <trisha@appsmith.com>
* Minor refactoring and fixed error messaging
Co-authored-by: Trisha Anand <trisha@appsmith.com>
* add elasticSearchPlugin
* Fix container startup in tests
* Add elasticsearch dependency
* Get plugin to a base working state
* Add templates and tests for all Document APIs
* Add support for bulk queries
* Add test and template for bulk operations
* Use rich form for action configuration
* Add test API for ElasticSearch
* Use rich form's values for plugin execution
* Add authorization header support
* Fix tests after config object use changes
* Add test for bulk requests with nd-json body
* Remove templates and minor refactoring
* Fix potential NPE with null body
Co-authored-by: Trisha Anand <trisha@appsmith.com>
* Add datasource validation for endpoint
* Wrap errors in AppsmithPluginException
Co-authored-by: Suman Patra <spatra@akamai.com>
Co-authored-by: Trisha Anand <trisha@appsmith.com>
* Add option to override server timezone for MySQL datasources
* Added test for server timezone override for MySQL
* Fix serverTimezone property config
* Implement caching for datasource structure
* Avoid using final fields so spring-data can load objects
* Use type variable for getStructure method
* Initial version with base structure for MySQL plugin
* Add tests for MySQL plugin structure
* Add sorting for keys in Postgres structure as well
* Show "primary key" instead of just "primary"
* Refactor to reduce inline magic strings
* Add a connection type variable for Plugin implementations
This type variable is intended to represent the type of the
connection object, if any, that the plugin will use. This will
help make the implementations more robust by leveraging Java's
type checking instead of rudimentary type casts over the
connection objects.
* Fix missing typevar usage
* Base interface for getting datasource structure for databases
* Add keys and constraints to structure for Postgres datasources
* Use connection from datasource context for computing structure
* Refactor context retrying into a separate method
* Add base datastructures for templates in entity explorer
* Fix spring circular dependency
* Add test for postgres datasource structure
* Generate column names and sample values for INSERT query
* Add LIMIT clause to generated SELECT query
* Fix tests for generated SELECT query
* Minor refactoring
* Fix aliases not showing up for MySQL actions
* Fix date column display for MySQL actions
* Fix datetime columns for MySQL actions
* Add support for timestamp and year data types for MySQL actions
* Fix column label for MySQL plugin
* Add tests for MySQL temporal data types
* Add tests for alias columns in MySQL and Postgres
* Added information to setup domain
* Fixed incorrect var reference
* Updated documentation message
* Updated env template
* updated template
* removed debug echo
* Updated Script
* Updated Text
* Removed option to connect to external mongo for fresh installs
Exit script of docker desktop is not installed
* Updated docker installation explanation
* added a question to skip custom domain if user is installing locally
* Inverted script question and
* Removed question to determine local installation
* Updated Popup CTA to Modal
Updated query templates to contain bindings
Added a message to display on API / Query errors
* updated styles for evaluated value
updated styles for auto complete
* added spaces to questions
Co-authored-by: Nikhil Nandagopal <nikhil@appsmith.com>
Keeping the function to read plugin resource generic for any type of resource. Now the parent function can also conditionally decide which plugin would require to read which resources from the filesystem.
Co-authored-by: Arpit Mohan <arpit@appsmith.com>
* Handle stale database connection from datasources
* Fix potential secondary case of stale connection error
* Fix Postgres to MySQL
* Move validity check timeout to a constant field
* Add test for recovery when stale connection error is thrown
The plugin class loader from Pf4J looks for jars under target/lib
to use to load any classes that are missing target/classes. This
PR adds the `maven-dependency-plugin` to copy the jars to this
folder automatically so the classes are available during
development.
This commit adds the Mysql plugin to the Appsmith server. We also add a migration to ensure that this plugin is installed by default for all existing organizations. The migration also adds the plugin details into the DB.
Also adding the test cases for mysql plugin.
Co-authored-by: Arpit Mohan <arpit@appsmith.com>
Co-authored-by: Hetu Nandu <hetu@appsmith.com>
Co-authored-by: Arpit Mohan <me@arpitmohan.com>
Co-authored-by: Nupur Singhal <nupursinghal@Nupurs-MacBook-Air.local>
* Dates from Postgres queries now show up as ISO date strings
* Use constant for date column name in PostgresPlugin
* Fix formats for more date-time types on Postgres
* Add support for time and timestamp data types in Postgres plugin
* Add support for timestamptz column data type in Postgres plugin
* Add support for interval data type in PostgresPlugin
* Move application configuration to be loaded from environment variables
* Remove unused sentry.properties
* Make missing value sentinel a constant and ignore all *.env files
* Removed now-used ACL properties
* Prefix RapidAPI environment variable with APPSMITH_
* Fix application properties not being loaded into static fields
* Remove application-test.properties file
* Add required env variables for test in GitHub
* Quote URLs for MongoDB and Redis in test config
* Change RAPIDAPI to RAPID_API in environment variable names
* Source .env file in the root of repo in start script
Add server-side templates support for plugins
Two main themes in this MR:
1. Add support for saving query templates as resource files inside a plugin. This has been done for both Postgres and MongoDB plugins, based on current template contents.
2. Loading of form JSON is now reactive. The blocking call has been moved inside a `Mono.fromSupplier`.
3. The loading of from JSON as well as the templates is cached and are loaded on-demand. This means that the templates are loaded once for a plugin through the lifetime of the server process, and that they are loaded only at the first time they are needed.
4. If loading of these resources fails, we try again when they are needed the next time.
See merge request theappsmith/internal-tools-server!385
This is to simplify the query pane on the frontend client. The client doesn't need to have separate interfaces for sql and non-sql plugins. All queries will be sent to the server in the form of a String that is parsed in different formats based on the plugin.
Also adding test cases for PostgresPlugin. Used TestContainers to simulate the postgres db in Docker inside the Java test itself. Very useful.
This is because the client sends us the data for form-data in the form of a list of Property fields with keys and values. This is consistent with how the client stores & renders headers & query parameters as well.
For MediaType application/x-www-form-urlencoded we will use the bodyFormData field.
2. Add to Page changes : Documentation object has been added in Action to handle the extra Template documentation for actions that have been imported from 3p marketplace
3. Added basic structure for rapid api plugin by copy pasting the rest api plugin
2. Add to Page changes : Documentation object has been added in Action to handle the extra Template documentation for actions that have been imported from 3p marketplace
3. Added basic structure for rapid api plugin by copy pasting the rest api plugin
