Fix internal server error in case of ARRAY type in MySQL (#17702)
This commit adds support for handling array like data in MySQL
- If the data type identified by the client side is of type ARRAY it will be treated as STRING as MySQL doesn't have support for ARRAY data type
* fix:Add array datatype to execute request
* feat: Consume and store type of array elements in Param class (#16994)
* Append param instead of clientDataType in varargs (#16994)
* Refactor common data type handling w.r.t newer structure (#16994)
This commit takes care of the following items:
- It minimizes the number of usage to the older stringToKnownDataTypeConverter method
- Modifies the existing test cases to conform to the newer structure
- Marks stringToKnownDataTypeConverter method as deprecated to discourage further use
* Remove comma delimited numbers from valid test cases (#16994)
* Fix extracting clientDataType from varargs in MySQL (#16994)
* Pass param as a dedicated parameter in json smart replacement (#16994)
* Remove varargs from json smart replacement method (#16994)
* Move BsonType to mongoplugin module (#16994)
* Introduce NullArrayType and refactor BsonType test cases (#16994)
* Add new test cases on numeric string with leading zero (#16994)
* Refactor test case name (#16994)
* Add comment on the ordering of Json and Bson types (#16994)
* Add comment on the ordering of Json and Bson types (#16994)
* Add NullArrayType in Postgres and introduce postgres-specific types (#16994)
* Add data type test cases for Postgres and change as per review comments (#16994)
Co-authored-by: ChandanBalajiBP <chandan@appsmith.com>
* Changes to testDatasource interface method and archive flow
* Tests for plugin level testDatasource implementations
* Added test for refreshing cache on deleting datasource
* Modified warnings to errors in logs
* Fixed test
* Fixed test
* feat: Implement data type handling in MySQL (#16621)
With this implementation we can now achieve the following under prepared statement:
- Ability to distinguish between null object and "null" string
- Ability to distinguish values like {{"098765"}} and {{098765}}. The former is identified as a string and the latter is identified as an integer
* feat: Add unit test cases on data type handling in MySQL (#16621)
* chore: Move MySQL specific types to a separate class (#16621)
* chore: Remove import shortening (#16621)
* Fix testStructure test case to have loose coupling with the order of tables (#16621)
* Fix: Add missing client-side data types in params in a few test cases (#16621)
* Fix query in test case (#16621)
* Fix test cases and add small refactoring (#16621)
* Refactor assertion with a check on Optional (#16621)
* additional cypress test cases for mysql
* updated test case object
* update for failing test cases
* mysql failure point fix
* mysql false spec failure point fix
* fix flacy test cases
* flacky JSON test cases
Co-authored-by: ChandanBalajiBP <chandan@appsmith.com>
Co-authored-by: Aishwarya UR <aishwarya@appsmith.com>
* BugFix: Fix for mongoPlugin smart substitution of regex attributes, Fixes: https://github.com/appsmithorg/appsmith/issues/11880
Changes:
MongoPlugin.java
MongoPluginTest.java
* Added test for decimal numbers and made changes to regex
* code formatting: Formatted some test cases in MongoPluginTest.java
* feat: Add a service layer to get AppsmithType from client-type (#16508)
This commit takes care of the following things
- It creates a service layer to get the target AppsmithType from the client-side data type and the evaluated value
- This service layer is currently not consumed by any plugins
- A full JUnit test suit for MySQL-specific types
* feat: Remove FloatType and add comments for better understanding (#16508)
* feat: Add FallbackType and covert the DataTypeService to a util class (#16508)
This commit takes care of these
- Introduce FallbackType as a separate type
- Convert the DataTypeService to a util class
- Add java doc around the methods in DataTypeService class
* feat: Rename DataTypeService to DataTypeServiceUtils (#16508)
On macOS, when running a MySQL query, we see the following error:
```
java.lang.ClassCastException: class io.netty.channel.kqueue.KQueueEventLoopGroup cannot be cast to class io.netty.channel.EventLoopGroup
```
This is because of a `LinkageError`, caused because these classes exist both in the parent `server` module classpath, as well as in this plugin's classpath separately. To fix this, we removed this dependency from the server module.
But then, this caused a problem with how the `DNSResolver` class was being, well, resolved by the application class loader, vs the plugin class loader. This is a macOS specific problem, and doesn't affect other operating systems.
For this, we add just this package as a dependency to the MySQL plugin.
* feat: Add server-side base data types (#15784)
This commit takes care of the following two things
- It adds the server-side base data types which are considered common data types across plugins
- It adds a few MySQL specific types
* feat: Add MySQLBooleanType and refactor MySQLDateTimeType with camelCase (#15784)
* feat: Move MySQL specific types to mysqlPlugin folder (#15784)
* feat: make AmazonS3 plugin error messages more readable. #8664
* added curly braces in if/else block
* modified according to PR review comments
* modified according to PR review comments
* This commit introduces readable errors for S3 Datasource: https://github.com/appsmithorg/appsmith/issues/8554
It changes three files:
*AmazonS3ErrorUtils.java
*AmazonS3PluginTest.java
*AmazonS3ErrorUtilsTest.java
This commit has been tested:
*manual testing
*Junit testing
* This feature-commit introduces readable errors for S3 Datasource: https://github.com/appsmithorg/appsmith/issues/8554
It changes three files:
*AmazonS3ErrorUtils.java
*AmazonS3PluginTest.java
*AmazonS3ErrorUtilsTest.java
This commit has been tested:
*manual testing
*Junit testing
* added new line in the AmazonS3ErrorUtilsTest.java
* * Ommited unused imports in following files:
* AmazonS3ErrorUtils.java
* AmazonS3PluginTest.java
* AmazonS3ErrorUtilsTest.java
* Moved test functions
* combined instance checking for AmazonServiceException and its subclass
* removed unused imports from AmaxonS3UtilsTest.Java
Co-authored-by: somna <somnathdasadhikari@gmail.com>
* fix: Use admin as default database when no Mongo DB is specified
* fix: Make default db compulsary instead
* Added back check during execution per review
* Json field spec fix
Co-authored-by: Aishwarya UR <aishwarya@appsmith.com>
* Modify control type for MongoDB Collection field
* Make more form control changes across datasources
* Fix cypress errors
* Fix typescript error
* Fix more cypress tests
* Fix failing MongoDBShoppingCart spec
Co-authored-by: Aishwarya UR <aishwarya@appsmith.com>
## Description
This PR fixes an issue where a potentially malicious user can connect to disallowed hosts from the Elasticsearch plugin within Appsmith. This is because Elasticsearch client SDK is a HTTP interface underneath the hood.
## Type of change
- Bug fix (non-breaking change which fixes an issue)
## How Has This Been Tested?
- Junits for the following:
- create datasource with disallowed host
- validate datasource with disallowed host
- test datasource with disallowed host
## Checklist:
- [x] My code follows the style guidelines of this project
- [x] I have performed a self-review of my own code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my feature works
- [x] New and existing unit tests pass locally with my changes
## Description
Fixes issue for checking for invalid hosts even when there are redirects in the Rest API plugin.
## Type of change
- Bug fix (non-breaking change which fixes an issue)
## How Has This Been Tested?
- Junit test
## Checklist:
- [ ] My code follows the style guidelines of this project
- [ ] I have performed a self-review of my own code
- [ ] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [ ] My changes generate no new warnings
- [ ] I have added tests that prove my fix is effective or that my feature works
- [ ] New and existing unit tests pass locally with my changes
* This commit changes two things:
* Pom.xml --> ES- Restclient version to 7.17.5 from 7.9.2
* ElasticSearchPlugin.java ---> changed the testDatasource method to expect a 200 instead of a 200 or 404 HTTP status code
This commit fixes a high-priority bug with github issue https://github.com/appsmithorg/appsmith/issues/14909
* it can now negotiate around a 403 forbidden error as it does try to send head only to user provided URI
This commit has been tested:
* manually for CRUD at local with ES version 7.9.2
* manually for CR at user provide aws instance running 7.9.3 on docker
* |
| BugFix:
|
| This commit changes two files:
|
| * ElasticSearchPlugin.java --> added feature to distinguish between unauthorized and not found datasources.
| * ElasticSearchPluginTest.java ---> added testcases to verify the unauthorized and not found test cases
|
| This commit adds features on top of previous commit https://github.com/appsmithorg/appsmith/issues/14909
| * this commit adds the feature of more readable error messages while testing the elasticsearch datasource. i.e wrong endpoint errors and unauthorized issues
|
| This commit has been tested:
| * Junit
| * manually
* removed leftover comments from the parent commit and updated response texts for better readability
* removed wildcard imports from appsmith-plugins/elasticSearchPlugin/src/test/java/com/external/plugins/ElasticSearchPluginTest.java
* update driver
remove connection closure when not required
* add connection pool
remove ssl options
* got working with postgres driver
* use Redshift driver instead of postgres
* updated JUnit TC
added comments
minor refactor
* add comment
cleanup
* update default port
* Added support for self signed certificate during OAuth2 flows, server side changes
* fix: authentication.useSelfSignedCert key added
* Merging from release
* Fixed issue with dependencies, plus bug with using ssl
* Cypress test attempt 1
* Cypress test attempt 2
* Cypress test attempt 3
* Clean up
* Review comments
* Enabled compression again
Co-authored-by: Aman Agarwal <aman@appsmith.com>
Upgrades vulnerable dependencies in all plugins except for MySQL. That one is still failing and I'll fix it in a separate PR. Issue #14475
Co-authored-by: Nayan <nayan@appsmith.com>
Co-authored-by: Anagh Hegde <anagh@appsmith.com>
* fix: Enable parsing dates before Jan 1, 1970 in MongoDB
This commit takes care of three things
1. It enables the application to parse dates before Jan 1, 1970 in case of MongoDB
2. JUnit test
3. Cypress test
* fix: Enable parsing dates before Jan 1, 1970 in MongoDB
This commit takes care of three things
1. It enables the application to parse dates before Jan 1, 1970 in case of MongoDB
2. JUnit test
3. Cypress test
Co-authored-by: Nidhi <nidhi.nair93@gmail.com>
* fix: Switched Google Sheets JSON to local directory
* fix: Fixed placeholder text for insert row object
* fix: Enabled server side caching of plugin resources again
* Client changes 1
* add DSL functionality
* Temp commit for refactoring changes
* Do I even know what I'm doing here?
* chore: Second GS layout
* Update: Visibility conditional outputs for schemas
- Added the output from conditional outputs for schema children too
* Update: Entity selector visibility control
- Added logic for controlling visibility of sub components via the JS expressions system
* Update: Passing disabled prop to toggle button
* Update: Passing disabled prop to toggle btn
* Update: Styled component for toggle button
- Added disabled styles based on the disabled prop sent to the toggle form view JSON button
* Update: configProperty role in Entity Selector
- Removed dependance of the configProperty of the entity selector children to it's parent component
* Update: type of placeholder key
- Made placeholder key from form config JSON to accept either string or an object
- Earlier only string was accepted
- This is for pagination component
* Update: Added placeholder control for pagination
* Client changes 1
* add DSL functionality
* Do I even know what I'm doing here?
* fix: updated uqi forms ui, clubbed JS switch button to ads, updated tooltip design
* fix: updated tooltip component for wrong ui on entity explore
* temp triggers
* fix: updated uqi forms ui, clubbed JS switch button to ads, updated tooltip design (#12395)
* fix: updated uqi forms ui, clubbed JS switch button to ads, updated tooltip design
* fix: updated tooltip component for wrong ui on entity explore
* fix: updated tooltip ui, where condition placement, sort by ui
* temp form data access logic
* fix: updated sorting type width ui
* fix: updated ui for spacing, width and text issues
* Update: Type for tooltip of UQI forms
- Added option to send an object to the tooltipText object.
- This allows for composite components like pagination to have tooltips for each sub component
* Update: tooltip for pagination component
- Added handling to parse the tooltip for multiple components.
- This allows for composite components like pagination to have tooltips for each sub component
* Update: Type cast for tooltip component
- Made the content passed to tooltip component as a string only
* Update: Fixed tooltip component CSS
* Update: Dropdown option component
- Added a tooltip wrapper to each option
- This is to show on hover text like disabled state
* fix: updated ẇhere clause broken ui for condition
* Add: functions to check and extract expressions
- Loop through the formConfig and find any keys that have a value that is bindable
- Used pre defined regex to check if value is a moustache binding
* Add: Types for evaluated form configs
- Added types for the form configs to be evaluated and their output post eval
* Add: Flow to run the form config
- Run the form config and update the result to the redux state
* Update: Name of the type for formconfigs
- Updated since it was clashing with a component of the same name
* Add: Function to enforce config type checks
- This is done so that the improper configs can be weeded out and the rest of the form can be shown
* Add: Function to update evaluated config
- Added option to update the config if it's values needed evaluation
* Add: Type check for schema sections
* Update: Error handling for invalid control type
- We were throwing an exception till now, changed it to a warning text
* Add: Exposed tooltip for dropdown option disabled state
* Update: switch to json mode functionality
- Added logic to convert data to a string rather than an object when the first switch to JSON mode happens
* Update: Added key to tooltip for dropdown options
* Trigger API modification
* Add: function to fetch default trigger URL
* Update: Made URL optional in dynamic trigger config
* Update: Dynamic trigger API call
- Made the API call for dynamic triggers have URL as optional field
- Added type check to the response of the API call
* Update: resp type for trigger APIs
* Update: Moved code to utils folder
- Moved functions for UQI form eval processing to utils file
* Update: passing original controltype to JS switch
* Update: config for JSON editor mode
- Updated the config to have different options for JSON mode depending on the original control type
* Update: Connected line numbers flag to config
* Revert: CSS changes for tooltip
* Refactor: Removed consle
* Add: type for the config of dynamic values
* Add: Feature to evaluate config for triggers
* Refactor: fix type check errors
* fix: dropdown ui width with text alignment
* Update: fixed selector for dynamic values
* Update: selector call for fetchDynamicValues
* Add table header index prop for columns selector
* migration partial commit
* migration partial commit
* Refactor: removed unused import
* Update: reused function for checking dynamic value
* Update: removed unused import
* Fix format JSON issues
* Retrieve binding paths from entity selector components
* Fixes 6 remaining issues with UQI implementation
* Fix dropdown issues
* Fix dropdown height issues and fixes triggering of APIs when option is deselected
* Migration changes
* Fix QA generated UQI issues
* Fix projection component height and route change logic
* Fix multi select dropdown placeholder text issue and json stringify issue with switching view types
* Reset entity type value when command value changes
* Test changes
* Review comments
* Moved migrations around
* Corrected import statement
* Added JSON schema migration
* Updated schema version
* perf improvements and filter dropdown options feature
* Fix Code mirror component config for toggleComponentToJson input fields.
* Fix prettier issues
* fix prettier issues
* Fix style issues as a result of the merged conflicts
* Fix failing test case
* Fixed a few other flows (#14225)
* Fixed a few other flows
* Review comments
* Fix generate CRUD, fix evaluation of dynamic bindings and fix various styling issues.
* More fixes (#14367)
* Factor in the root formconfig parent key.
* Fix flickering issues, and evaluatedFormConfig issues
* fix: Teeny bugs (#14455)
* Teeny bugs
* Added previous functionality as is
* Improvements in the way we fetch dynamic values
* Fix stringiification issue and cyclic dependency issues
* Resolve projection component values deletion
* Resolve merge conflicts and fix prettier issues
* fix: Tsc issues
* Fix property pane connection navigation
* updating ee locator
* updating inputfield locator
* dropdown locator update
* Merge conflict not properly resolved.
* Fix s3 spec
* Fix Mongo Spec
* Fix some more tests
* fix: prevent cyclic dependency when switching to js mode (#14668)
* add delete events for change from array to string in diff
* add test to assert absence of cyclic dependency error when switching to js in switchgroup widget
* Assert that evaluation is not disabled when no cyclic dependency happens
* Cypress test preparations for google sheets and form controls
* Fixed a few test errors (#14874)
* Add: unit tests for uqi UI updates
- view type tests
- conditional output extraction
- processing conditional output to handle view/enabled state of the component
* Add: completed isValidFormConfig test
* Update: improved tests for update config
- These tests cover the functionality to update a section config after it's components are done evaluating
* Fix failing cypress tests and cyclic dependency issue
* Fixes some more tests
* Fixed migration of row objects (#14896)
* Bumped the version of design system package
* Update: reverted change to EE selector
* Fix deletion pointer
* Update: selector for js on load spec
- Synced with changes related to ADS dropdown
* Fix mongoDBShoppingCart spec
* Remove comments
* Fix: mongo shopping cart test failures
* fix: mongo shopping cart spec
* Dummy push to retrigger vercel
* fix: mongo shopping cart spec
* Update MongoDBShoppingCart_spec.js
* fix: removed unused click away
* dummy commit
* Update: moved helper functions to separate file
* Add: added tests for saga functions
- Worked on testing for
- extractFetchDynamicValueFormConfigs
- extractQueueOfValuesToBeFetched
* Add if check for queueOfValuesToBeFetched
* Resolve review comments
* Empty-Commit
Co-authored-by: Irongade <adeoluayangade@yahoo.com>
Co-authored-by: Ayush Pahwa <ayush@appsmith.com>
Co-authored-by: Aman Agarwal <aman@appsmith.com>
Co-authored-by: Ayangade Adeoluwa <37867493+Irongade@users.noreply.github.com>
Co-authored-by: Aishwarya UR <aishwarya@appsmith.com>
Co-authored-by: Favour Ohanekwu <fohanekwu@gmail.com>
Co-authored-by: Albin <albin@appsmith.com>
* fix: Consider zip compression info from API response header (#13515)
This commit fixes two things
- If certain API has Accept-Encoding header then Appsmith will take care of this before rendering response to the client
- Used MediaType.includes function for more extensive matching of the requsted API's content type
* fix: Add unit test on consuming API with gzip encoded response (#13515)
This commit takes care of two things
- Unit test on consuming an API which has response being encoded in GZip format
- Set default content-type to "text/plain" in case the content-type is not present in API response
* fix: Add Cypress test on consuming API with gzip encoded response (#13515)
This commit takes care of the following
- Cypress test on consuming an API which checks if it can read any property from the response sent from the API server encoded in GZip format
* made test classes public
* added surefire compatible with junit 4
* Revert the changes of tests class accessibility
* Fix test failures
* test case fixes
* fixed NPE
* Fixed failing tests
Co-authored-by: Anagh Hegde <anagh@appsmith.com>
Co-authored-by: Nidhi <nidhi.nair93@gmail.com>
* Fix 11141-Added JDBC override parameter for resolving Cast Errors with timestamp
* Firestore Where condition Unit Test for Arrays
* Code review comments update
* removed two unrelated file somehow added here from the previous PR
* Code review comments update V2
* Code review Comments Fix - Second
* Removing the frontend Changes related to dropdown width
Co-authored-by: Leo Thomas <leoweb2010@gmail.com>
* DBRef fix with codec
* Added code comments regarding custom codec implemented
* Added Unit Test for Mongo DBRef
Co-authored-by: Leo Thomas <leoweb2010@gmail.com>
Co-authored-by: Sumit Kumar <sumit@appsmith.com>
* fix: Using form url encoded content type for token requests
* fix: Using form url encoded content type for cc as well
* fix: Conversion logic for expires in field
* Currently S3 plugin datasource requires the region field to be non-empty for it to work (when configured with other as the datasource type from dropdown). However, this field is not required for many S3 offerings. Hence, making it non-mandatory.
* feat: Add Array type to MultipartFormDataType variants
and handle every item in list as a part in multipart request
* fix: add array options
* fix: rename file to array value
* fix: parse Array from json text
* fix: reformat the code and remove `System.out.println`
* fix: testParseMultipartArrayDataWorks is more accurate now
Co-authored-by: Segun Daniel Oluwadare <dodanieloluwadare@gmail.com>
Co-authored-by: Aishwarya UR <aishwarya@appsmith.com>
* update option label
* add minIO as a separate datasource option
add datasource connection changes to support minIO without SSL and when using CA signed certificates
* fix cypress tests
* This PR adds the JSON smart substitution feature to Firestore plugin actions.
* This smart substitution will work exactly the same way as it works for REST API actions.
* Currently, smart substitution only gets applied to the body section of Create, Set, Add or Update commands as this is the only input field that expects a JSON input type.
* It can be toggled On/Off via the Settings tab.
* It is set to On by default. However, data migration has been added to set it to Off for all existing Firestore actions so that they are not impacted by this PR's changes.
* fix: remove asterisk from non-mandatory fields in firestore query
* making one false
* fix: remove asterisk from non-mandatory fields in firestore query
* fix: remove asterisk from non-mandatory fields in firestore query
Co-authored-by: Hritik Kumar <devnamrits@Hritiks-Mac.local>
* shelf: start working on substitution for `ISODate`
* a draft for Sumit to check what I did wrong in my tests
* test: tests are working
* test: cleanup
* refactor: add replacementDataType parameter to DataTypeStringUtils.jsonSmartReplacementPlaceholderWithValue
* fix: remove quotes for Mongo Types work in single form, arrays and json.
* add handling for more use cases
add/update more TCs
* add comments
* add more comments
* add space (cosmetic change)
Co-authored-by: Sumit Kumar <sumit@appsmith.com>
* fix: if test datasource gets Unauthorized accessing admin database
It will try to access the default database.
* fix: fix broken test
* fix: cleanup unused imports
* test: make test more readable with better comments
* fix: 7794 Google Sheet Fails with Mixed Datatype
* resolved failing test cases after new implementation
* Added code comments to the changes
Co-authored-by: Leo Thomas <leoweb2010@gmail.com>
* WIP
* Refactoring HTTP Method & Content Type to be objects instead of arrays
TODO:
1. Set the default content-type for Get request to "None". Currently, it's raw
2. For None content-type, don't send the body field in the API request
* Almost working implementation for the None type
Currently, the body still gets sent in non-GET requests even if the None tab is selected.
* Adding object.freeze to prevent any modifications to HTTP_METHOD_ENUM
* WIP: Using enum & const for ts autocomplete
* working implementation for NONE type, apiContentType prop added to API actions
* WIP
* Refactoring HTTP Method & Content Type to be objects instead of arrays
TODO:
1. Set the default content-type for Get request to "None". Currently, it's raw
2. For None content-type, don't send the body field in the API request
* Almost working implementation for the None type
Currently, the body still gets sent in non-GET requests even if the None tab is selected.
* Adding object.freeze to prevent any modifications to HTTP_METHOD_ENUM
* WIP: Using enum & const for ts autocomplete
* working implementation for NONE type, apiContentType prop added to API actions
* adds apiContentType to actionConfiguration.formData object
* Handling apiContentType property in Rest API formData
* change apiContentType when user types content-type value and switches http method
* makes api editor as similar as possible to postman, project postman.
* Correcting the import in ApiEditorConstants
* Resolved all merge conflicts
* replay DSL functtionality
* removes unneccessary files from worker
* Fixes type declarations, naming e.t.c.
* fix server side merge conflicts
* fix client side merge conflicts
* fix failing cypress tests
Co-authored-by: Irongade <adeoluayangade@yahoo.com>
Co-authored-by: Ayangade Adeoluwa <37867493+Irongade@users.noreply.github.com>
* This PR removes the Expiry Duration field from the Read file command of S3 plugin.
* An Expiry Duration field got added to Read file command mistakenly some time post migration to UQI schema. This field did not exist with the read command before the migration and does not get used even now.
* Fixed issue with Firestore in fetching data while using numbers in the where condition.
* Other Fixes made in the where condition :
When using Boolean
When using Date
When Using Timestamp
When given space in the key
When Space given in the Value
* Add `getStructure` implementation for S3 to return a list of accessible buckets.
* Add command templates for S3.
- List files
- Read file
- Create file
- Delete file
* Remove getDatasourceMetadata implementation as getStructure method can be directly used now.
* This PR adds a limit field which sets the batchSize for aggregate command. This way the user can set the number of documents they want in their Mongo query result.
* limit is set to 10 initially. However, for pre-existing Mongo actions, migration has been added to set this field to 101 - which is the default value that has been used by Mongodb in the absence of any user set value. Ref: https://docs.mongodb.com/manual/tutorial/iterate-a-cursor/
* fix:9648 Mongo update All issue-In Progress
* fix: Mongo Multi update query fail fixes
* Added updated testcases that got affected by the code change
* Changes reverted as per review comments
- fix Mongo plugin's smart substitution feature to handle quotes around ObjectId in array. e.g. Initial replacement value: ["ObjectId(xyz)"] , final replacement value: [ObjectId(xyz)]
- unrelated: added version number to lombok dependency to stop build failures in IntelliJ.
* feat:9389 Add support to send arbitrary form values to oAuth server during Datasource authentication
* Changes related to Code Review Comments of PR 9650
* Changes implemented to set custom Key Value pairs
* Resolved Code Review comments
* Resolved Code Review comments
* fix/8924-Bulk Insert throwing error on empty data in google sheets new1
* fix/8924 Error in bulk insert empty data
* fix/8924 Code review fix for Google Sheet Bulk insert error
* fix/8924 Code review fix for Google Sheet Bulk insert error
* Fixed code review comments
* Methodconfig data changes since only empty Mono is being tested
* Port Firestore plugin to UQI schema
* This PR implements a non-functional requirement i.e. from an end user perspective there is no change in the features that the plugin provides. However, the underlying DSL to render and interpret the query interface has been updated.
* It also involves a migration change to port existing Firestore plugin actions to the new UQI schema.
* Added parsing of where condition to Condition format
* Refactoring the name of functions to denote old style before implementing UQI where clause
* Updated the error message for data type problem for mixed data tyeps
* Generating complex logical expression.
* Working version of filtering happening without refactoring of code
* Added where clause configuration in List files in a bucket command.
Not able to render due to some configuration issue.
* Untested code completion
* To be reverted. Ayush's changes.
* Tested where condition on S3 List
* Made AND the default option for where clause
* where clause working in case of no valid inputs provided.
* Added parallel test cases as that were existing for the old where clause
* Revert "To be reverted. Ayush's changes."
This reverts commit a0f9b72e241f0688b7ef07cea8c3017473423512.
* Making equality the default option in a new where clause for LIST command
* Added test cases incorporating the review comments.
* Updated the options for the where command in S3 plugin. Removed the comparison operators except equality, non equality and belonging (in and not in) operators.
* Added catching of exception while parsing the operator into known appsmith condition types
* Reusing objectmapper from BasePlugin instead of creating a new one here.
* Remove Region field from S3 datasource editor page for AWS S3, Upcloud, Digital Ocean Spaces, Dream Objects, Wasabi.
* Use SDK provided property for AWS S3 to delegate region selection to the SDK.
* Extract region info from endpoint URL for Upcloud, Digital Ocean Spaces, Dream Objects and Wasabi, since the SDK property does not work for these service providers.
* Removed some redundant checks from datasourceCreate that were already part of validateDatasource
* Fix show clause in list.json
The problem was that the pagination was designed & tested with only 2 pages. Firestore has a different function `limitToLast` which solves for previous page pagination commands.
* ArangoDB has two types of command - one that fetches a response and the other that updates a document. Currently, ArangoDB plugin treats all cmds like the one that returns a response, hence when update cmds are run then an empty response is returned to the client since there is nothing to fetch. This PR adds a check to fetch the response based on the type of cmd. Hence, for an update cmd the number of writes succeeded or failed would be returned as a response.
* Adding `APPSMITH_PLUGIN_MAX_RESPONSE_SIZE` which can be configured as an environment variable to increase plugin response size (currently only supported for Postgres Plugin)
* Updated error message + added the environment variable to the template docker env files for new installations
* Minor refactoring
* Partial code change to replace question mark with appsmith placeholder
* Working version
* Removing unnecessary code
* Added test case to assert that when evaluated value contains a "?", the replacements are still correct
* Added test case in Mongo Plugin as well for the same scenario
* Minor change in the language of the comment
* Added smart defaults for query in Count, Distinct & Find commands
* Added test cases for smart inputs for query and limit for find, and query for count and distinct
* This method removes the outermost quotes - single or double quotes - so that end users don't have to do it via javascript inside widget fields where they are meant to be bound.
* The input box label in the Firestore plugin action editor reads Document / Collection Path which is misleading since a document is nested inside a collection and not the other way round. Hence changing it to Collection / Document Path
* A fix had been introduced earlier (I had introduced it a few months back) to remove data type info keys from the raw response. However, the transformation was not getting applied to the Query action as the transformation was explicitly limited to few actions. This PR removes that check and allows the transformation for all actions as there does not seem to be any reason to withhold this transformation for other actions.
* The earlier fix also restricted this transformation to results that had specific keys. This PR removes this check as well as it does not seem to be required and further may lead to similar issues if any particular key gets missed.
* Some other redundant checks have been removed from the flow as the transformation function itself takes care of it as edge/base case.
* Earlier split method was used to segregate cmd and args which failed when multiple words inside a quoted string formed one argument. e.g. set key "my value" would produce set, key, "my, value" as the tokens when split method is used, which is not correct. This change introduces a regex that would create the following tokens: set, key, "my value"
* Added field to autogenerate values in DB column
* Added comment and refactor
* Autogenerate field update according to names instead of type for ArangoDB
* 1. For binary data in the response, base64 encoding the same before sending it out on the wire (instead of default conversion to string)
2. Added a new header which sets the data type of the response body.
* Minor code reformatting
* Fixed failing test cases
* The new header created should send the values in an array instead of a single value
* API to run DB query using plugin specified templates
* Included get spreadsheet metadata in get info method
* Added TCs for checking Spreadsheet info response
* Added error message for invalid datasources
* Authentication check for datasource modified to AuthenticationStatus field
* Adding support for explicit data type casting in postgres prepared statement
* Added text and int psql data types for support in explicit typecasting
* Documenting the code
The default charset is determined during virtual-machine startup and typically depends upon the locale and charset of the underlying operating system (#5788)
Co-authored-by: 王昆 <quincy@cloudtogo.cn>
* add support in Redis plugin to select database when creating db connection.
* refactor code to use URI string instead of constructor call, because of lack of appropriate constructor.
* add TCs.
* fix validate datasource.
* return hint message to user if the expected content type is json but the api response is not a valid json.
Co-authored-by: Apeksha Bhosale <7846888+ApekshaBhosale@users.noreply.github.com>
* Raw is now a command option instead of Form vs Raw
* Added database migration for migration raw input type to be of raw command type
* Fixed test case failure
* Minor cleanup
* add api key based and bearer token based authentication options to REST API datasource
* this change is ported to release from Feature: datasource changes for api key auth, bearer token auth, combine config. #4683
* Update commands have been merged into a single update command for both single and multi updates
* Added migrations to migrate any update one mongo form command to new update command
* Incorporated review comments
* POC for snowflake integration with Appsmith
This is just a rough version to see if the functionality works. Needs to be cleaned up & enhanced for production usage.
* Plugin with connection, monos, ds testing, validation, templates, structure and forms
* Better comments
* Default schema to public
* Changed logging destination, populated request in result and used thread group for datasource creation
Co-authored-by: Arpit Mohan <arpit@appsmith.com>
* Reduced verbosity of the command names for Insert, Find and Delete
* WIP : Smart substitution untested changes for mongo form
* Tested code for smart substitution working for mongo form fields
* Added test case for assert for smart substitution
* Correcting text for Mongo Form Insert command
* Getting Prepared Statement for Postgres, MySQL, and MS-SQL and Smart Substitution for Rest API plugins out of beta
* Fixed failing test case
* Fixed Ms-sql plugin test failure
* fix socket error
* fix default port
* remove invalid check for missing port, since default port would be supplied if user leaves the port field empty.
* Handling multipart form data in the CURL import flow
* Adding Cypress test for curl import with multipart/form-data
* Also fixing minor bug where the form values need not always be double-quoted
Because of the Java version not being explicitly set
in this plugin, the build of the whole project fails
sporadically in IntelliJ since it's trying to compile the
code as Java 8, which will obviously fail.
- return request parameters with type for debug tab.
- request params are stored in a RequestParamDTO object.
- a configProperty -> label map is cached in pluginService, which returns the param label as per the configProperty it is mapped to via editor.json file.
- This feature currently only works with prepared statements disabled.
* Initial scaffolding for comments CRUD APIs
* add actions
* add assets
* state management for existing comments and creating new
* add ui components
* add overlay comments wrapper to baseWidget
* add toggle comment mode button at editor header
* trigger tests
* Disallow commenting as someone else
* Add applicationId for comments
* lint
* Add overlay blacklist to prevent component interaction while adding comments
* Comment thread style updates
* Placeholder comment context menu
* Controlled comment thread visibility for making new comments visible by default
* Update comment type description
* Reset input on save
* Resolve comment thread button ui
* fix close on esc key, dont create new comment on outside click
* Submit on enter
* add emoji picker
* Attempt at adding a websocket server in Java
* CRUD APIs for comment threads
* Add API for getting all threads in application
* Move types to a separate file
* Initial commit for real time server (RTS)
* Add script to start RTS
* Fix position property
* Use create comment thread API
* Use add comment to thread API
* Add custom cursor
* Dispatch logout init on 401 errors
* Allow CORS for real time connection
* Add more logs to RTS
* Fix construction of MongoClient
* WIP: Real time comments
* Enable comments
* Minor updates
* Read backend API base URL from environment
* Escape to reset comments mode
* Set popover position as auto and boundary as scroll parent
* Disable warning
* Added permissions for comment threads
* Add resolved API for comment threads
* Migration to set commenting permission on existing apps
* Fix updates bringing the RTS down
* Show view latest button, scroll to bottom on creating a new comment
* Cleanup comment reducer
* Move to typescript for RTS
* Add missing server.ts and tsconfig files
* Resolve / unresolve comment
* Scaffold app comments
* Minor fixes: comment on top of all widgets, add toggle button at viewer header
* Reconnect socket on creating a new app, set connected status in store
* Retry socket connection flow
* Integration tests for comments with api mocks using msw
* Fix circular depependency
* rm file
* Minor cleanup and comments
* Minor refactors: move isScrolledToBottom to common hooks, decouple prevent interactions overlay from comments wrapper
* Use policies when pushing updates in RTS
* ENV var to set if comments are enabled
* Fix: check if editor/viewer is initialised before waiting for init action
* Add tests for comments reducer
* Revert "ENV var to set if comments are enabled"
This reverts commit 988efeaa69d378d943a387e1e73510334958adc5.
* Enable comments for users with appsmith email
* lint
* fix
* Try running a socket.io server inside backend
* Update comment reducer tests
* Init mentions within comments
* Fix comment thread updates with email rooms
* Minor fixes
* Refactors / review suggestions
* lint
* increase cache limit for builds
* Comment out tests for feature that's under development
* Add Dockerfile for RTS
* Fix policies missing for first comment in threads
* Use draftJS for comments input with mentions support
* fix fixtures
* Use thread's policies when querying for threads
* Update socket.io to v4
* Add support for richer body with mentions
* Update comment body type to RawDraftContentState
* fix stale method
* Fix mentions search
* Minor cleanups
* Comment context menu and thread UI updates
* revert: Scaffold app comments
* Yarn dependencies
* Delete comment using id api added
* Init app comments
* Add test for creating thread
* Api for delete comment with id
* Test comment creation response and policies
* Copy comment links
* Fix reset editor state
* Delete valid comment testcase added
* Delete comment TC : code refactor
* Don't allow creating comments with an empty body
* Pin comments WIP[]
* Ignore dependency-reduced-pom.xml files from VCS
* Cleanup of some dev-only files, for review
* Delete comment
* Update socket.io to v4 in RTS
* Pin and resolve comment thread object added in commentThread
* Pin and resolve comment thread object added in commentThread
* Update comment thread API
* Added creationTime and updationTime in comment thread response
* Added creationTime and updationTime in comment thread response
* Added human readable id to comment threads, fallback to username for null name in user document
* Refactor
* lint
* fix test, rm duplicate selector
* comment out saga used for dev
* CommentThread viewed status, username fallback for getName=null, username field added in pin & resolve status
* lint
* trigger tests
Co-authored-by: Shrikant Sharat Kandula <shrikant@appsmith.com>
Co-authored-by: Abhijeet <abhi.nagarnaik@gmail.com>
* Only consider headings till value grid size, but consider all values till this size
* Switched from null to blank for consistency
* Whoops, we don't do null row objects anymore
- return helpful error message when get structure fails for mongo db due to lack of read permission. The message returned is: Appsmith has failed to get database structure. Please provide read permission on the database to fix this.
- any get structure error is also generically being caught by the getStrucuture method in DatasourceStructureSolution.java
- error msg and status code is now returned as part of DatasourceStructure object.
- ErrorDTO is used to store error info.
- ErrorDTO definition is moved to appsmith-interfaces package so that it can be used all packages.
- Other changes:
- stop logging PLUGIN_GET_STRUCTURE_ERROR externally, as this error is mostly triggered by credentials error or -lack of read permission.
- start logging PLUGIN_GET_STRUCTURE_TIMEOUT_ERROR externally, because this is not a user error.
* WIP : Migration of data type for smart substitution configuration
* Fixed the migration code. Also reverted editor.jsons back to equality check after migrations of existing actions
* Only fetching actions for plugins which belong to supported plugins.
* Setting a default value of "false" for all the actions which don't have a valid configuration for smart substitution.
* Minor code formatting
* Minor language change in comment
* Removed debug logs
* cherry pick -make new
* revert to enable fix from release
* attempt to hook into existing datasource editor
* gSheets plugin skeleton from Rest API
* Changes for database migration
* fix for auth code
* separate it out
* action page loads!
* add to explorer
* create action from datasource
* Editor JSON WIP
* working query form
* Editor JSON WIP
* import to
* fix toast message
* redirect from datasource and editor pages
* fix onboarding
* fix imports and constants
* refactor form out
* refactor queryForm
* Merge branch 'release' into feature/google-sheets
* Merge branch 'release' into feature/google-sheets
* initial values from settings, editor and form
* Check
* remove dangling code around lightTheme
* Safety net
* remove class
* try mouseover solve
* force click
* changes from review
* fix action form name on import
* Merge branch 'release' into feature/google-sheets
* minor cleanup
* Merge branch 'release' into feature/google-sheets
* WIP
* Google sheets changes
* Merge conflicts
* Merging and fixes, needs refactoring
* Check
* Merge branch 'release' into feature/google-sheets
* Fixed tests
* Add cloud services env variable
* Clean up saga
* Clean up
* Refactoring
* Deleted svg file
* Minor fixes
* Modified design to allow behaviour in google sheets methods (#3486)
* Modified design to allow behaviour in google sheets methods
* Review changes
* Removed sysout
* Added handling of edge cases with table data
* Merge branch 'release' into feature/google-sheets
* Fixes
* Fixes
* Added validations
* Improved tests
* Removed extraneous injected bean
* Review changes
* Fixed bug with method
* Changes to Google sheets plugin's request and response structures (#3692)
* Method changes
* Removed logging
* Renaming options
* Reverting pom version
* Modified type of collection variables, fixed errors
* Converted row offset field to one that supports dynamic bindings
* Review changes
* List SAAS plugin type actions under lightning menu apis (#3820)
* list saas plugin type actions under lightning menu apis
* combine saas plugin type actions in the other sub menu of lightning menu
Co-authored-by: Hetu Nandu <hetunandu@gmail.com>
* Fix merge issues
* Prettified query editor and a few fixes w/ ux
* Test fixes
* Reformatting request
* code for REST added (#3876)
Co-authored-by: hetunandu <hetu@appsmith.com>
* Renamed body to row object
* Renamed placeholder for range
* Renamed range heading
* Modifications to handle range semantics
* Use spreadsheet Url instead of id
* Ordering of methods
* Removed logging
* Add tests for Dynamic text controls
* Add tests for url helpers
* Fix coverage config
* Nevermind
* Interface changes
* There is no body here
* Yay to hints
* Delete row field is separately handled as row index
* placeholder support (#4001)
* Fixed tests, typos and creating new sheets with random rows
* Switched to using 'rowIndex' throughout
* binding path added for query input field (#4016)
* - Fixed QA bugs (#4032)
- Split delete sheet into two
- Removed dynamic query input types from hidden keys
* Proper exceptions
* Removed extra logging
* Throw exception if update method does not match any of the columns
* Same for bulk update
* Zero-indexed delete row
* I'm a space bound rocket ship
* Logic to register installations with cs (#4062)
* Logic to register installations with cs
* Clean up
* Casting to string
* Checking to see if this makes the test pass
* Added an extra null check
Co-authored-by: Piyush <piyush@codeitout.com>
Co-authored-by: hetunandu <hetu@appsmith.com>
Co-authored-by: Hetu Nandu <hetunandu@gmail.com>
Co-authored-by: Apeksha Bhosale <7846888+ApekshaBhosale@users.noreply.github.com>
* WIP : Type migration from String to Object for value
* Migrating config from string to boolean for prepared statement.
Handled error for already stored actions where PS config is stored as String and not Boolean.
* Added dependencies to be added to dependencyMap on client to Postgres, MSSQL, MySQL and Mongo plugins
* Added Dependency config for API
* Fixed the test case
Maven 3.8 now blocks HTTP repositories and only allows HTTPS. However, the version redshift jdbc dependency we are using connects to a separate repository that runs on HTTP. The latest version has switched to using HTTPS instead.
So this PR upgrades redshift dependency from version 2.0.0.1 to 2.0.0.4.
Ref: https://maven.apache.org/docs/3.8.1/release-notes.html#cve-2021-26291
- Return data type list along with action execution result so that the data can be displayed in the correct format.
- Handles these three data types for now : table, json, raw.
- If a plugin has already assigned the data type, then common handler is not used.
- Mongo plugin assigns the data types in the plugin specific flow i.e. does not use the common data type parse / assign method.
- To address review comments:
- added a new enum ActionResultDataType for data types. It is consumed by ParsedDataType. A list of ParsedDataType is returned by ActionExecutionResult
- parsed data in parallel using streams.parallel().
- add title to action execution errors to improve user experience
- all errors in AppsmithPluginErrors
- action execution related errors in AppsmithError : NO_CONFIGURATION_FOUND_IN_DATASOURCE, INVALID_ACTION, INVALID_DATASOURCE, INVALID_DATASOURCE_CONFIGURATION - as these are the likely errors in an action execution flow (excluding plugin specific flow) that might result from faulty action / datasource configuration by a user.
- title is returned as part of ActionExecutionResult.
- title is set in ActionExecutionResult on failure during action execution.
- In response to review comment - added a new BaseException class, so that both AppsmithException and AppsmithPluginException extend this base class. Also, refactored code to introduce setErrorInfo() function in ActionExecutionResult class to set its attributes.
- Some unrelated cleanup:
- catch PoolInitializationException for Postgres plugin and return AppsmithPuginException
- catch MongoTimeoutException and return AppsmithPluginException
* Feature : JSON Smart Substitution in Mongo Plugin
* Added BSON data type for Mongo substitution. Added test case
* Minor comment added
* Minor variable re-naming and correcting failing test cases.
* REST API : Escaping special characters in string before smart substitution
* Incorporated review comment
- Allow users to define key path for FieldValue.delete() and FieldValue.serverTimestamp() values.
- delete() value is only valid for update operation, as Firestore does not seem to support it for any other ops.
- serverTimestamp() is valid for all operations excluding get and delete operations.
* added empty byte array for request body to fix aws signed url signature match problem as detailed here - https://github.com/aws/aws-sdk-java/issues/2205
Co-authored-by: Bernard Worthy <>
* Smart Substitution Debugging - Adding appsmith data type to each parameter in response's request object for execute
* Incorporated review comments.
* Fixed failing test cases.
* Fixed MS-SQL test failures.
* WIP : Adding timestamp datatype in Prepared Statement
* Added simple tests for commonly used data types' identification
* Added plugin level test cases for timestamp with Prepared Statement to Postgres
