* Include version information in backend builds
* Enable manual trigger of server workflow
* Add a dummy input field for manual trigger
* Fix secret checking in github actions
* Disable cron scheduling of sync job
* Build docker image on pushing a version tag
* Fix duplicated id and invalid id in release workflow
* Don't try to login to Docker
* Avoid download progress outputs from Maven
* Add version information for building client
* Calculate version number only once
* Enable push to DockerHub after building images
* Use Docker username from GitHub secrets
* Fix Docker username in tags when building image
* Use different secret for org name
* Minor refactoring in server workflow
* Update client build workflow to use version
* Enable manual triggering of client workflow
* Set snapshot version for server builds
* Push to docker for all branches (temp)
* Undo temp change to push all branches to DockerHub
* Use Docker org from secrets in client.yml
* Fix missing version reference in client.yml
* Remove debug command in dockerfile
* Save release notes in a resource file in Docker image
* Fix query serialization to get release notes
* Get releases of current repo instead of hard-coded repo
* Fix variable quoting for repo variables
* Exclude draft and prerelease nodes from image
* Fix call to any in release notes processor
* Fix syntax error in release notes script
* Implement API to get new release count and info
* Add missing ReleaseNotes component
* Have the release workflow run after a release is created
* Build server after generating release notes
* Change release trigger to "released"
* Change release trigger to "published"
* Change release trigger to released, edited and deleted
* Use JS script to get release notes, take 1
* Filter drafts and prereleases in script
* Fix syntax error in ES6
* Write release notes to file
* Create parent directory before writing release notes
* Log cwd in release notes script
* Log pwd along with release-notes content
* Handle case where working directory is incorrect
* Remove shell based release notes generator
* Don't show error when Sentry config is missing
* Check for sentry auth token to enable Sentry
* Carry build's exit code over to CI
* Mark out build result and add a note about it
* Add a small test to verify new versions computation
* Remove incorrect test assertion
* Remove generation of release notes file
* Connect to cloud services to fetch release notes data
* Fix missing runner for test class
* Handle missing cloud_services base URL
* Fix test failures due to missing mocks
* Enable sync-ee cron job
* Revert build.sh as there's no real change
* Add API to update release notes viewed version for users
* Fix prettier line-length errors
* Create UserData model for info unrelated to auth
* Fix field name calls
* Ensure we have a userId before setting userData
* Add tests for setting version number in UserData
* Include instanceId when fetching release notes
* Disable signup API via environment variable
* Allow signup for invited users, even if disabled publicly
* Add test for signup when signup is disabled
* Run invite flow tests when signup is disabled
* Revert status annotation in signup API endpoint
* Remove unused tests on invite flow
* Change signup disabled error message
* Fixed unreferenced actions not being triggered for execute on load
* WIP : Have to commit some minor cosmetic code formatting changes
* Comments and fix for action ordering
* modified test case
Co-authored-by: Trisha Anand <trisha@appsmith.com>
* Increased logs to debug future connection leaks.
* Fetch a connection from the pool only if a query exists.
* Minor comment added.
* Minor rewrite
* Code formatting.
* Update app/server/appsmith-plugins/postgresPlugin/src/main/java/com/external/plugins/PostgresPlugin.java
Co-authored-by: Shrikant Sharat Kandula <shrikant@appsmith.com>
* Added hikari cp pool stats to get database structure function as well.
Co-authored-by: Shrikant Sharat Kandula <shrikant@appsmith.com>
1. Get rid of cypress test failures which asserts a certain order till the cypress test case is fixed and unblocks the frontend team.
2. Uses a bulk API to fetch all the pages instead of fetching each page sequentially over different network calls with the database.
* Zipping flux with mono leads to mono not getting repeated and only the first emitted value of flux is processed further. This is leading to only a single page getting emitted in a multi page application.
* Added a test case to assert this for a multi page application
* All newly created applications now have a default published state.
* Added a test case to assert a new application comes up with default published states for both application and the default page.
* Function name changed.
* When trying to fetch a published version of a page when the page has not been published, throw a 404.
* WIP : Not working
* Adding the skeleton for Firestore integration
* Adding the datasource & query editor forms
Also adding the database changelog for the firestore plugin
Commenting out the firestore.close() connection because that causes issues with multiple Firestore tenants running in the same JVM.
* Adding the code for fetching the structure of collections from Firestore
* Use single document path field for Firestore
* Fix potential NPE when datasource destroy timeouts
* Work in progress on collection level ops for Firestore
* Get documents in a collection now works
* Add collection level querying support
* Mild refactoring
* Fix NPE when some fields are missing
* Hide clientJSON as a password field for Firestore
* Make collection level querying reactive
* Make reactive
* Validate before connecting
* Add tests for all supported methods in Firestore
* Fix forms for Firestore with hidden fields
* Hide limit and order by fields when not needed
* Restore log entry deleted by mistake
* Use S3 URL for Firestore/Firebase logo
* Add comments detailing why some code is commented
* Make parsing JSON reactive and fix subscribe calls
* Fix reactive scheduler
Co-authored-by: Arpit Mohan <arpit@appsmith.com>
* Making postgres plugin blocking statements run on elastic threadpool.
* Moved the blocking calls to check for the connection validity, etc. also inside the Mono.fromCallable
Instead of using compound index, using single indices to ensure that both fetch actions by app id (during load of actions in edit/view mode) and fetch actions by page id (during update layout) are indexed queries.
* Doing a deep copy during clone application instead of updating the original application which may have been causing a concurrency bug.
* Ensuring that once the clone application flow is triggered, the flow completes eventually even if the client cancels the request before completion.
* Cloned application would not be public.
* Added parametrized Application constructor
* Removed lombok all args constructor
* Optimized import
* Incorporated review comments :
1. Updated the constructor for creating the application
2. Added a test case to assert that if during cloning of an application the flow gets cancelled, the cloning would still complete and ensure that the application created is sane.
Also adding the APPSMITH_SEGMENT_CE_KEY as a build arg to the server Dockerfile. This will be used to set the environment variable in the Docker image itself.
* Experimenting with spring metrics, prometheus
* Added prometheus config in application.properties.
* Added a few more configurations for spring metrics to improve performance tooling.
* Added basic authentication to allow prometheus to scrape our monitoring data in an authenticated fashion.
* Only exposing prometheus end point at this junction instead of other metric endpoints as well.
* Fixed test case which failed because of actuator instantiating bean.
* Report selective errors to rollbar.
1. Assign an action attribute to each AppsmithError enum item. Take action based on the action attribute.
* Fixing the condition for organization name being empty
* Report selective errors to rollbar.
1. Assign an action attribute to each AppsmithError enum item. Take action based on the action attribute.
Co-authored-by: Sumit Kumar <sumitsum@Sumits-MacBook-Pro.local>
Co-authored-by: Arpit Mohan <arpit@appsmith.com>
* Fixed timeout error message, added new type of error for plugin timeouts
* Switched error code and added test for a secondary StaleConnectionException
* [Do Not Merge]Brought down cloud migration time from 122 minutes to 9.5 minutes
* Removed commented code.
TODO : Figure out if we want to change the migration order numbers once the release environment is reverted back to pre original migration data.
* Added plugin id for the migrated action.
* Renamed the createIndex functions for new action and new page and added a drop collection command before creating the index. This ensures that if any developer has pulled code in the last two days, this migration does not negatively impact them
* Adding host:port validation checks to ES plugin & Redis plugin
Also correcting the assertions in the RedisPluginTest so that the error message on test failure is accurate.
* Removing the endpoint validation from datasourceServiceImpl
Moving the endpoint validation to the plugin implementation classes because there are databases that require complete HTTP URL in their configuration. Checking for http/https at the platform level affects the UX for the user & plugin developer when using a new integration. Hence, all plugins must implement their own client/server validations in their own implementations
* Adding tests to assert hostname validation in mysql & postgres plugins
* Bug Fix : Reading the datasource configuration incorrectly from actionDTO instead of datasource object.
* Added test case to assert that datasource configuration and action configurations are not null when action execute is called.
* Introduced new page which stores the published and unpublished pages as separate.
* Mid level commit to save the state.
* Parity of new page repository with old page repository (custom functions)
* WIP : Delete a page. This requires changes across application structure as well.
* Added publishedPages construct inside application to store the pages in the deployed view as well as isDefault so that the same changes (delete or isDefault) in unpublished view doesn't alter these fields for the published application
* Parity reached with PageService.
* Minor ActionService refactor to remove unnecessary code.
ApplicationPageService, LayoutActionService, LayoutService use the new page service to fetch the pages
Minor corrections in fetching the page from new page service in tests
* New save function which sets the PageDTO for unpublished page and then saves the new page into repository.
* Migration of page service functions to new page service functions across other services/tests/controller
* Finished migrating all the page service functions to the new page service functions
* Application Service Tests have been fixed.
* All the existing test cases are working now.
* Publish application implemented to store published pages as well. Added a basic test case to check that published pages is being set and that page's publishedPageDTO is being set accordingly.
* Minor TODOs added to add test cases for published application.
* A few tests to ascertain that published application page fields (deleted, isDefault) does not get changed when these statuses are changed for a page in edit mode.
* Added a new controller end point to fetch application in view mode.
* Added new endpoint for fetching an application in view mode on the client.
* Bug fix where get application in view mode API was not getting called.
* Fixed the get page names by application & archive pages which have been deleted in edit mode during publishing of application.
* During delete page, if a page was never published and it was deleted during edit, delete the entire page instead of just deleting the unpublished PageDTO
* Minor formatting.
* Non working client side code to fetch page list using view mode.
* revert unnecassary changes and streamlined view and edit actions
* Fix missed import
* Fixed a bug where if a page is not published, it should not be returned in view mode in list of page names api.
* Fixed update for a page which was not working in integration test.
* ActionDTO added.
* Solidified the new action structure.
* Migration added for NewAction index creation and NewAction per Action insertion in the database.
* Basic file structure added the new repository, custom repository, service, etc.
* Delete OldPage.java
* Repo functions added - TODO : Haven;t handled the published/edited views
* Helper functions added to convert Action to NewAction and vice-versa. Removed unused currentUserMono usage.
* Create & update action functionality added.
* Execute Action refactored. Removed dry run specific code.
* Repository migrated to handle new data structure. Execute action refactored to no longer support dry runs of actions.
* TODO added for special handling of change view of application to handle edge cases of pages/actions which either exist in published mode but don't exist in unpublished mode or vice versa.
* Migrated finding on load actions from spring repository to custom repository.
* In view mode, now actions are being fetched by application id directly instead of first fetching application and then using the page ids, fetching the actions. This reduces the db calls from 2 to 1 per fetch actions in view mode api call.
* Delete action and get all actions (used in edit mode on the client side) implemented.
* Updated CollectionService and ActionCollectionService to use the new action service instead of the old one.
* LayoutActionService refactored to now use the new service functions.
* ActionController now no longer used ActionService. The remaining service functions have been migrated to the new action service.
* Refactor across ACL code for addition/removal of policies during addition/removal of users to organization, making app public, refactor for services like policy utils, item service, etc.
* Removed the last of action repository and action service and replaced with new action repo and new action service.
* Compile and run time issues fixed. The server is coming up without any spring dependency errors.
* WIP in fixing fetching actions by page id.
* Finally!!! Fixed the fetch actions (both published and unpublished actions) by page id repository function.
* Fixed create action bug where null datasource in published actiondto (inside newly created action) leads to error.
* Fixed the execute action issues :
1. Removed the dry runs from the tests
2. Fixed the null pointer error in variable substituted action and datasource configurations.
* 1. Fixed the custom action repository field names.
2. Fixed the data structures used in ExamplesOrganizationClonerTests
* Fixed countByDatasourceId repository function which was querying the actions incorrectly.
* Fixed the clone example organization error where the id of the action was not getting updated in the page correctly. Yay!
* Fixed post merge compilation failure.
* Fixed more compilation time failures in ActionServiceTest
* Fixed failing test case for fetching actions in view mode.
* Minor changes to resolve merge changes and incorporate in the new refactored code.
* 1. Fixed compile time errors on Client code.
2. Fixed fetching of actions in view mode by application id. The repository function did not need name parameter. Removed the same.
* [Integration Testing Error Fix] : Added a new test case for refactor action name.
* Instead of fetching actions in the page, mistakenly used the base service which was fetching all the actions in the repository, barring none which led to the name refactor being blocked even though no action in the current page exists with the new proposed name,
* Added delete functionality to action service.
* Minor code cleanup
* Adding viewMode to action execution
* Replacing action with actionId.
* 1. Bug fix for deletion of unpublished action. In case of never published action, the entire action should be deleted. In case an action was published, only the unpublished action should be deleted.
2. In case of DB actions (external datasources), only the bare minimum fields should be stored inside the action (datasource id and datasource plugin id). The other fields should not be duplicated across code.
* Fixed yarn build compilation issues.
* Update app/server/appsmith-server/src/main/java/com/appsmith/server/controllers/ActionController.java
Co-authored-by: Arpit Mohan <mohanarpit@users.noreply.github.com>
* Changed the API path for GET applications in view mode. Some minor code formatting.
* Incorporated review comments.
* Some more unnecessary code removed.
* Instead of returning Page, now the interface object between client and server for Pages is PageDTO
* Migrated Page and Action to PageDTO and ActionDTO
Fixed the compilation issues.
TODO : Fix the test compilation issues.
* Fixed compilation time issues with all the tests by migrating Page and Action to PageDTO and ActionDTO respectively
* Action Controller and Page Controller no longer extend Base Controller. All the required functions have now been implemented and no base line API end points are being re-used from the base.
* Test case fixes.
* Bug Fix : Updating an action was not updating execute on load. Fixed the data flow leading to the error.
* Deprecating Page and Action domain objects. This is to ensure no new code is written with these till we remove this old code.
* Cloned example applications are now published before returning. This is to ensure that the applications are in ready to view mode when the new user signs up.
* Added a function comment to expand on the usage of new param introduced.
* When cloning a page, new actions were not being stored. Added that. Also updated the clonePage test to assert that the actions are also cloned when the pages are cloned.
* Updated a Api call
* removed extra slash
Co-authored-by: Hetu Nandu <hetunandu@gmail.com>
Co-authored-by: Satbir Singh <satbir121@gmail.com>
Co-authored-by: Arpit Mohan <mohanarpit@users.noreply.github.com>
Co-authored-by: nandan.anantharamu <nandan@thinkify.io>
1. Fixing the build by excluding the slf4j-api from redis-plugin pom.xml
2. Adding the editor.json and form.json for the query pane & datasource pane.
3. Adding array handling in the Redis response by feeding all the output into a "result" key
* Added DynamoPlugin to integrate with DynamoDB
* Connection management for DynamoPlugin
* Process action from a single JSON object as input
* Moved ListTables command to separate class
* Add missing UI definition files
* Use reflection to build request objects for AWS SDK
* All DynamoDB actions are now run purely by reflection
* Serialize responses to JSON
* Remove some unused/commented code and some refactorings
* Add non-working test with generic dynamodb container
* Added working test for ListTables action
* Added test for PutItem action
* Add tests for get and update actions
* Added comments and some refactoring
* Removed unused Command class.
* Added more tests for map to SDK object conversion
* Add templates
* Add option to set an endpoint override
* Fix Dynamo plugin tests when ~/.aws is missing
* Add documentation link for DynamoDB plugin
* Fix validation checks
* Remove debug log
* Added comments on expected query structure
* Fix incorrect error condition with incorrect request format
* Add comment for converting action to method name
* Error out if region is missing
* Remove unused throws declaration
* Use rich form for action configuration
* Removed templates
* Use PNG for logo images
* Use PNG logos for all plugins
* Avoid hard-coded field names
* Change logo to PNG only for ElasticSearch for now
* Wrap errors in AppsmithPluginException
* Typo
Co-authored-by: Trisha Anand <trisha@appsmith.com>
* Fix changeset order number
Co-authored-by: Trisha Anand <trisha@appsmith.com>
* Minor refactoring and fixed error messaging
Co-authored-by: Trisha Anand <trisha@appsmith.com>
* add elasticSearchPlugin
* Fix container startup in tests
* Add elasticsearch dependency
* Get plugin to a base working state
* Add templates and tests for all Document APIs
* Add support for bulk queries
* Add test and template for bulk operations
* Use rich form for action configuration
* Add test API for ElasticSearch
* Use rich form's values for plugin execution
* Add authorization header support
* Fix tests after config object use changes
* Add test for bulk requests with nd-json body
* Remove templates and minor refactoring
* Fix potential NPE with null body
Co-authored-by: Trisha Anand <trisha@appsmith.com>
* Add datasource validation for endpoint
* Wrap errors in AppsmithPluginException
Co-authored-by: Suman Patra <spatra@akamai.com>
Co-authored-by: Trisha Anand <trisha@appsmith.com>
* Returning validation failures while creating organization
* Added unit tests
* Changing the name of test security config
Co-authored-by: nitesh261193 <nitesh261193@gmail.com>
This is because this `cacheResponse` field is only used for auto-complete on the UI. The user can still run the action manually and get the auto-complete to work. The downside to sending the `cacheResponse` field in the JSON response is that for large responses, the client times out as our response times increase to 20 secs. Hence removing this for now.
In future, we'll extract the JSON schema structure from the response body and send that to the client for populating auto-complete.
* Implement caching for datasource structure
* Avoid using final fields so spring-data can load objects
* Use type variable for getStructure method
* Initial version with base structure for MySQL plugin
* Add tests for MySQL plugin structure
* Add sorting for keys in Postgres structure as well
* Show "primary key" instead of just "primary"
* Refactor to reduce inline magic strings
* Add a connection type variable for Plugin implementations
This type variable is intended to represent the type of the
connection object, if any, that the plugin will use. This will
help make the implementations more robust by leveraging Java's
type checking instead of rudimentary type casts over the
connection objects.
* Fix missing typevar usage
* Base interface for getting datasource structure for databases
* Add keys and constraints to structure for Postgres datasources
* Use connection from datasource context for computing structure
* Refactor context retrying into a separate method
* Add base datastructures for templates in entity explorer
* Fix spring circular dependency
* Add test for postgres datasource structure
* Generate column names and sample values for INSERT query
* Add LIMIT clause to generated SELECT query
* Fix tests for generated SELECT query
* Minor refactoring
* Add migration to fix incorrect action IDs for onLoad
* Remove redundant comments
* Fix migration to work with published onLoad actions as well
* Only update the action ID when correcting onLoad action IDs
* Fix migration name for correcting action IDs
Co-authored-by: Trisha Anand <trisha@appsmith.com>
* Fix migration id for correcting action IDs
* Reformat code
Co-authored-by: Trisha Anand <trisha@appsmith.com>
* Get list of application IDs to be cloned as examples
* Fix tests so that apps to be cloned are explicit
* Add migration to set applicationIds for example cloning
* Fix example org cloning tests
* Add a node.js script to dump examples organization
This dump file is used to setup initial examples organization
in a self-hosted setup.
* Remove unused variable
* Add npm run shortcut for dump script
* Start with new controller for assets
* Progress on uploading logo image
* Saving and loading organization logo assets now works
* Remove existing logo asset before saving a new one
* Fix 500 when uploading logo for the first time
* Fix URL in response for uploading logo image
* Add test for uploading logo for ogranization
* Mild refactoring in tests
* GET API actions recognized as page load actions should also be updated as executeOnLoad actions.
* Update action during updateLayout only if executeOnLoad is not set or is false.
* WIP : First draft of clone applications
* Cloning of application now works correctly.
* Fixed the failing test case for page cloning.
* Added test case to assert the clone of application successfully.
* Minor function renaming to avoid confusion.
* Minor code refactoring to remove duplicate code.
1. Fixed the bug where policies are being set to empty during organization update.
2. Added email field in organization.
* Updated the test case for update organization to assert the organization policies being present post the update.
We are currently getting the *list* of all pages, updating the
`isDefault` fields inside, and then saving the whole *list* of
all pages. If a new page got added to that list in the DB during
this process, that page would be lost. This commit fixes this
problem.
This race condition was causing tests for cloning applications
to fail *sometimes*.
* Working version of cloning page given page id. The clone is created inside the same application and is in unpublished state.
* Added a test case for Clone Page feature
* Incorporated review comments.
* Don't always automatically create a blank personal org
It will still be created, only if a example template organization
has not been configured in the system.
* Name examples organization same as personal organization
* Minor fixes in tests
* Remove unused userService in clone tests
* During add role to an organziation, the application was only inheriting from subset of the organization permissions. Generalized this code to ensure that this doesnt happen again in the future when more permissions are introduced. Refactored some code as well.
* Introduced new permissions for making app public and inviting users.
* WIP : Returning a subset of roles depending on the current user's role
* Added code and tests for generating hierarchical children given a role name
* Get all invite roles now returns roles depending on the current users permissions (aka - admin can invite users for all roles, developers can invite users for roles develoeprs and below)
* When invite api is called, check if the user has correct permissions ( invite permission) and check if the role for which the users are bieng invited is allowed (aka the current user has the permission to invite for the said role)
* Introduced a new permission at application level which is inherited from manage:organizations. This new permission gives the administrator of the organization to make any application public .
* Incorporated review comments.
* Minor code changes.
* Added migration script to add the new policies introduced as part of this new share modal changes.
* Null pointer exception handled due to improper data in the database.
* Integration testing bug fixes.
* Correcting the Docker image for Appsmith server for Github actions test
* Also fixing the bug where the datasource config is not returned properly for restapi-plugin
Co-authored-by: Arpit Mohan <arpit@appsmith.com>
