## Description
1. Added frontend and backend custom OTLP telemetry to track execute
flow
2. Updated end vars in client side code to match with server sdk
intialisation code.
#### PR fixes following issue(s)
Fixes#28800 and #28805
#### Type of change
- Chore (housekeeping or task changes that don't impact user perception)
#### How Has This Been Tested?
- [x] Manual
- [ ] JUnit
- [ ] Jest
- [ ] Cypress
>
>
#### Test Plan
> Add Testsmith test cases links that relate to this PR
>
>
#### Issues raised during DP testing
> Link issues raised during DP testing for better visiblity and tracking
(copy link from comments dropped on this PR)
>
>
>
## Checklist:
#### Dev activity
- [x] My code follows the style guidelines of this project
- [x] I have performed a self-review of my own code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [ ] I have added tests that prove my fix is effective or that my
feature works
- [ ] New and existing unit tests pass locally with my changes
- [ ] PR is being merged under a feature flag
#### QA activity:
- [ ] [Speedbreak
features](https://github.com/appsmithorg/TestSmith/wiki/Guidelines-for-test-plans#speedbreakers-)
have been covered
- [ ] Test plan covers all impacted features and [areas of
interest](https://github.com/appsmithorg/TestSmith/wiki/Guidelines-for-test-plans#areas-of-interest-)
- [ ] Test plan has been peer reviewed by project stakeholders and other
QA members
- [ ] Manually tested functionality on DP
- [ ] We had an implementation alignment call with stakeholders post QA
Round 2
- [ ] Cypress test cases have been added and approved by SDET/manual QA
- [ ] Added `Test Plan Approved` label after Cypress tests were reviewed
- [ ] Added `Test Plan Approved` label after JUnit tests were reviewed
Changes
add rate limit on login and signup APIs
add annotations to support rate limit on controllers which can be
configured per API.
refactor SecurityConfig
implementation details
uses bucket4j for rate limiting
uses redis as a backend for distributed rate limiting
fixes: https://github.com/appsmithorg/appsmith-ee/issues/234
**Changes**
* add rate limit on login and signup APIs
* add annotations to support rate limit on controllers which can be
configured per API.
* refactor SecurityConfig
**implementation details**
* uses bucket4j for rate limiting
* uses redis as a backend for distributed rate limiting
Bumps
[org.bouncycastle:bcprov-jdk18on](https://github.com/bcgit/bc-java) from
1.72 to 1.74.
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/bcgit/bc-java/blob/main/docs/releasenotes.html">org.bouncycastle:bcprov-jdk18on's
changelog</a>.</em></p>
<blockquote>
<!-- raw HTML omitted -->
<!-- raw HTML omitted -->
<!-- raw HTML omitted -->
<p><!-- raw HTML omitted --><!-- raw HTML omitted -->2.1.1 Version<!--
raw HTML omitted --><!-- raw HTML omitted -->
Release: 1.76<!-- raw HTML omitted -->
Date: 2023, July 29th</p>
<!-- raw HTML omitted -->
<p><!-- raw HTML omitted --><!-- raw HTML omitted -->2.2.1 Version<!--
raw HTML omitted --><!-- raw HTML omitted -->
Release: 1.75<!-- raw HTML omitted -->
Date: 2023, June 21st</p>
<!-- raw HTML omitted -->
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Commits</summary>
<ul>
<li>See full diff in <a
href="https://github.com/bcgit/bc-java/commits">compare view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
You can disable automated security fix PRs for this repo from the
[Security Alerts
page](https://github.com/appsmithorg/appsmith/network/alerts).
</details>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
## Description
As of now the CS API does not have signature verification which can lead
to data tampering for CS API response. This PR adds the method to add
signature verification for CS API responses.
Corresponding PRs:
CS: https://github.com/appsmithorg/cloud-services/pull/1023
#### PR fixes following issue(s)
Fixes https://github.com/appsmithorg/cloud-services/issues/1037
#### Type of change
- New feature (non-breaking change which adds functionality)
## Testing
#### How Has This Been Tested?
- [ ] Manual
## Checklist:
#### Dev activity
- [x] My code follows the style guidelines of this project
- [x] I have performed a self-review of my own code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [ ] I have added tests that prove my fix is effective or that my
feature works
- [ ] New and existing unit tests pass locally with my changes
- [ ] PR is being merged under a feature flag
#### QA activity:
- [ ] [Speedbreak
features](https://github.com/appsmithorg/TestSmith/wiki/Guidelines-for-test-plans#speedbreakers-)
have been covered
- [ ] Test plan covers all impacted features and [areas of
interest](https://github.com/appsmithorg/TestSmith/wiki/Guidelines-for-test-plans#areas-of-interest-)
- [ ] Test plan has been peer reviewed by project stakeholders and other
QA members
- [ ] Manually tested functionality on DP
- [ ] We had an implementation alignment call with stakeholders post QA
Round 2
- [ ] Cypress test cases have been added and approved by SDET/manual QA
- [ ] Added `Test Plan Approved` label after Cypress tests were reviewed
- [ ] Added `Test Plan Approved` label after JUnit tests were reviewed
This PR adds support to Appsmith server to be able to connect to
ElastiCache Redis with cluster mode turned on. When the
`APPSMITH_REDIS_URL` is set to `redis://...`, the current default
behaviour is preserved. But if it is set to `redis-cluster://...`, then
we setup a pooled connection with cluster mod enabled.
## Description
This PR enables the Sentry error logging for Appsmith cloud server.
Related slack thread:
https://theappsmith.slack.com/archives/CPQNLFHTN/p1685513675580309?thread_ts=1684307194.980669&cid=CPQNLFHTN
#### PR fixes following issue(s)
Fixes https://github.com/appsmithorg/cloud-services/issues/920
#### Type of change
- Bug fix (non-breaking change which fixes an issue)
#### How Has This Been Tested?
- [x] Manual
<img width="1721" alt="Screenshot 2023-06-28 at 11 19 27 AM"
src="https://github.com/appsmithorg/appsmith/assets/41686026/66d1d677-4bf8-4e30-9120-6cca6c834d42">
## Checklist:
#### Dev activity
- [x] My code follows the style guidelines of this project
- [x] I have performed a self-review of my own code
- [ ] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [ ] My changes generate no new warnings
- [ ] I have added tests that prove my fix is effective or that my
feature works
- [ ] New and existing unit tests pass locally with my changes
- [ ] PR is being merged under a feature flag
#### QA activity:
- [ ] [Speedbreak
features](https://github.com/appsmithorg/TestSmith/wiki/Guidelines-for-test-plans#speedbreakers-)
have been covered
- [ ] Test plan covers all impacted features and [areas of
interest](https://github.com/appsmithorg/TestSmith/wiki/Guidelines-for-test-plans#areas-of-interest-)
- [ ] Test plan has been peer reviewed by project stakeholders and other
QA members
- [ ] Manually tested functionality on DP
- [ ] We had an implementation alignment call with stakeholders post QA
Round 2
- [ ] Cypress test cases have been added and approved by SDET/manual QA
- [ ] Added `Test Plan Approved` label after Cypress tests were reviewed
- [ ] Added `Test Plan Approved` label after JUnit tests were reviewed
## Description
> Add file locking using redis with retry. Helpful when multiple users
are trying to perform git actions in the Appsmith UI.
Fixes#10856Fixes#14366
## Type of change
- Bug fix (non-breaking change which fixes an issue)
## How Has This Been Tested?
- Test A
- Test B
## Checklist:
- [x] My code follows the style guidelines of this project
- [x] I have performed a self-review of my own code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [ ] My changes generate no new warnings
- [ ] I have added tests that prove my fix is effective or that my
feature works
- [ ] New and existing unit tests pass locally with my changes
---------
Co-authored-by: Aishwarya UR <aishwarya@appsmith.com>
Co-authored-by: Sidhant Goel <sidhant@appsmith.com>
Co-authored-by: Parthvi Goswami <parthvigoswami@Parthvis-MacBook-Pro.local>
Co-authored-by: System Administrator <root@Parthvis-MacBook-Pro.local>
## Description
Import application flow is a multi-stage process where we touch all the
mongo DB collections required to store the application in DB, which
today includes pages, actions, JSObjects, etc. When the flow is complete
then only we can say the application is successfully imported and DB
won't have any stale objects stored. But in a negative scenario where
the flow might break because of some unknown exceptions, objects stored
earlier become stale and reside in DB forever. With this PR we are
establishing the infra for transactions to avoid saving stale DB
objects. This is achieved by a rollback mechanism in case an exception
is thrown in the middle of execution.
Note: Since transactions are built on concepts of logical sessions they
require mecahnics (like oplog) which are only available in replica set
environment.
You can always convert a standalone to a single noded replica set and
transactions will work with this one node.
[https://docs.mongodb.com/manual/tutorial/convert-standalone-to-replica-set/](https://docs.mongodb.com/manual/tutorial/convert-standalone-to-replica-set/?_ga=2.53712296.1453502529.1669195955-2065030687.1664009965)
```
For local dev earlier we used to run the single node without replica set. Henceforth if we want to take the advantage of transactions please follow the steps to run mongod single node in local with replica set:
- Upgrade the MongoDB version to 4.4 or higher if it’s 4.2 or lower (https://www.mongodb.com/docs/manual/release-notes/4.4/#std-label-4.4-upgrade)
- Close the mongoDB instance running in your local
- Start the mongoDB in replica set mode and initiate the replica set
- mongod --port 27017 --dbpath <path/to/db> --replSet <replica-set-name> && mongo --eval “rs.initiate()”
- One can use following commands to check replica set status:
- mongo appsmith
- rs.status()
- By this time you should have the mongo running with replica set
```
<img width="1788" alt="Screenshot 2022-07-01 at 10 31 27 PM"
src="https://user-images.githubusercontent.com/41686026/176944386-f9d94715-c0cf-4900-93b7-f73647132d60.png">
This also means mongodb connection string used in env file will now
include the replica-set name if one wants to leverage transactions:
`mongodb://localhost:27017/appsmith?replicaSet={replica-set-name}`
Fixes https://github.com/appsmithorg/appsmith/issues/14543
## Type of change
- Bug fix (non-breaking change which fixes an issue)
- This change requires a documentation update
## How Has This Been Tested?
> JUnit
> Manual test
## Checklist:
- [x] My code follows the style guidelines of this project
- [x] I have performed a self-review of my own code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [ ] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
---------
Co-authored-by: Anagh Hegde <anagh@appsmith.com>
Co-authored-by: Aishwarya UR <aishwarya@appsmith.com>
## Description
TL;DR: This PR introduces metrics logging using native Spring support
for Micrometer. It includes a docker-compose to set up all the required
parts of this observability stack in the local environment as well.
In order to make use of this stack, please navigate to
`utils/observability` and execute the following command:
```
docker-compose up -d
```
The set up comes bundled with a default Grafana dashboard that can be
accessed at localhost:3001. Please feel free to switch the mapping ports
around in the docker-compose file.
This dashboard currently shows all http requests (sampled at 0.1 by
default), and the server side implementation has introduced some minimal
tracing for the `/api/v1/action/execute` endpoint. This means that you
can use the trace id from http server requests for this endpoint to
delve deeper into the spans exposed in this flow.
In case you would like to send trace information to another service,
please make use of the `APPSMITH_TRACING_ENDPOINT` variable. To override
the default sampling rate in your local (to say, 1), you can set that as
the value for the variable `APPSMITH_SAMPLING_PROBABILITY`.
Fixes#19153
## Type of change
- Chore (housekeeping or task changes that don't impact user perception)
## How Has This Been Tested?
- Manual
### Test Plan
No testing required, only needs regression after merge.
## Checklist:
### Dev activity
- [x] My code follows the style guidelines of this project
- [x] I have performed a self-review of my own code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [ ] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [ ] PR is being merged under a feature flag
---------
Co-authored-by: Sumesh Pradhan <sumesh@appsmith.com>
- Add connection pool to MySQL
- Fix JUnit TC failures due to Spring upgrade
- Fix Cypress TC failures due to change in the MySQL plugin code
- Remove `Preferred` SSL option
This upgrade takes care of our move to JDK 17, Spring Boot 3.0.1 and a
few other security upgrades along the way.
Fixes#18993
TODO:
- [x] Check CI changes for Java 17
- [x] Check vulnerability report
- [x] Mongock needs an upgrade
- [x] Add JVM args at all possible places for exposing java.time module
- [x] Add type adapters everywhere / use the same config for type
adapters everywhere
On macOS, when running a MySQL query, we see the following error:
```
java.lang.ClassCastException: class io.netty.channel.kqueue.KQueueEventLoopGroup cannot be cast to class io.netty.channel.EventLoopGroup
```
This is because of a `LinkageError`, caused because these classes exist both in the parent `server` module classpath, as well as in this plugin's classpath separately. To fix this, we removed this dependency from the server module.
But then, this caused a problem with how the `DNSResolver` class was being, well, resolved by the application class loader, vs the plugin class loader. This is a macOS specific problem, and doesn't affect other operating systems.
For this, we add just this package as a dependency to the MySQL plugin.
* Insert git application metadata to application object, for git sync/version control feature there is a need to save the application in the structured format
* Added changelog to index application collection with git branch and remoteurl
* Rehydrate and dehydrate applications among different instances
* Added serailisation purpose while exporting the application to make distinction between version control and regular import-export flow
* Git file operations moved to appsmith-git module
* Delete json reference from directory for deleted resource in DB
* Added git authentication in application metadata
* Replace String with Path to handle file path in git file structure
* Initial scaffolding for comments CRUD APIs
* add actions
* add assets
* state management for existing comments and creating new
* add ui components
* add overlay comments wrapper to baseWidget
* add toggle comment mode button at editor header
* trigger tests
* Disallow commenting as someone else
* Add applicationId for comments
* lint
* Add overlay blacklist to prevent component interaction while adding comments
* Comment thread style updates
* Placeholder comment context menu
* Controlled comment thread visibility for making new comments visible by default
* Update comment type description
* Reset input on save
* Resolve comment thread button ui
* fix close on esc key, dont create new comment on outside click
* Submit on enter
* add emoji picker
* Attempt at adding a websocket server in Java
* CRUD APIs for comment threads
* Add API for getting all threads in application
* Move types to a separate file
* Initial commit for real time server (RTS)
* Add script to start RTS
* Fix position property
* Use create comment thread API
* Use add comment to thread API
* Add custom cursor
* Dispatch logout init on 401 errors
* Allow CORS for real time connection
* Add more logs to RTS
* Fix construction of MongoClient
* WIP: Real time comments
* Enable comments
* Minor updates
* Read backend API base URL from environment
* Escape to reset comments mode
* Set popover position as auto and boundary as scroll parent
* Disable warning
* Added permissions for comment threads
* Add resolved API for comment threads
* Migration to set commenting permission on existing apps
* Fix updates bringing the RTS down
* Show view latest button, scroll to bottom on creating a new comment
* Cleanup comment reducer
* Move to typescript for RTS
* Add missing server.ts and tsconfig files
* Resolve / unresolve comment
* Scaffold app comments
* Minor fixes: comment on top of all widgets, add toggle button at viewer header
* Reconnect socket on creating a new app, set connected status in store
* Retry socket connection flow
* Integration tests for comments with api mocks using msw
* Fix circular depependency
* rm file
* Minor cleanup and comments
* Minor refactors: move isScrolledToBottom to common hooks, decouple prevent interactions overlay from comments wrapper
* Use policies when pushing updates in RTS
* ENV var to set if comments are enabled
* Fix: check if editor/viewer is initialised before waiting for init action
* Add tests for comments reducer
* Revert "ENV var to set if comments are enabled"
This reverts commit 988efeaa69d378d943a387e1e73510334958adc5.
* Enable comments for users with appsmith email
* lint
* fix
* Try running a socket.io server inside backend
* Update comment reducer tests
* Init mentions within comments
* Fix comment thread updates with email rooms
* Minor fixes
* Refactors / review suggestions
* lint
* increase cache limit for builds
* Comment out tests for feature that's under development
* Add Dockerfile for RTS
* Fix policies missing for first comment in threads
* Use draftJS for comments input with mentions support
* fix fixtures
* Use thread's policies when querying for threads
* Update socket.io to v4
* Add support for richer body with mentions
* Update comment body type to RawDraftContentState
* fix stale method
* Fix mentions search
* Minor cleanups
* Comment context menu and thread UI updates
* revert: Scaffold app comments
* Yarn dependencies
* Delete comment using id api added
* Init app comments
* Add test for creating thread
* Api for delete comment with id
* Test comment creation response and policies
* Copy comment links
* Fix reset editor state
* Delete valid comment testcase added
* Delete comment TC : code refactor
* Don't allow creating comments with an empty body
* Pin comments WIP[]
* Ignore dependency-reduced-pom.xml files from VCS
* Cleanup of some dev-only files, for review
* Delete comment
* Update socket.io to v4 in RTS
* Pin and resolve comment thread object added in commentThread
* Pin and resolve comment thread object added in commentThread
* Update comment thread API
* Added creationTime and updationTime in comment thread response
* Added creationTime and updationTime in comment thread response
* Added human readable id to comment threads, fallback to username for null name in user document
* Refactor
* lint
* fix test, rm duplicate selector
* comment out saga used for dev
* CommentThread viewed status, username fallback for getName=null, username field added in pin & resolve status
* lint
* trigger tests
Co-authored-by: Shrikant Sharat Kandula <shrikant@appsmith.com>
Co-authored-by: Abhijeet <abhi.nagarnaik@gmail.com>
