Two goals.
1. Move encryption/decryption implementation to `interfaces` module.
Today, only the interfaces are in `interfaces` module, not the
implementations. We need the implementation also to be in `interfaces`.
We need this for solving encryption/decryption in the `pg` branch.
2. Make the encryption and decryption functions be static, and not
depend on Spring's DI system. We need this to be available _before_
Spring's DI kicks in, during runtime annotation processing by Hibernate,
so we need it to be just static functions, and not depend on Spring
havin initialized.
This also means that we have to read the env variables directly. The
`application.properties` is loaded by Spring, and since we want this
information before Spring intializes, we'll need to read the env
variables directly.
⚠️ There's conflicts and additional changes needed on EE for build to
pass. **DO NOT MERGE** unless the author is known to be available. EE PR
also passed all at https://github.com/appsmithorg/appsmith-ee/pull/4282.
**/test all**
<!-- This is an auto-generated comment: Cypress test results -->
> [!TIP]
> 🟢🟢🟢 All cypress tests have passed! 🎉🎉🎉
> Workflow run:
<https://github.com/appsmithorg/appsmith/actions/runs/9467180088>
> Commit: 402785da8df5aa6b4eeec9955421ce1ff7af305f
> Cypress dashboard url: <a
href="https://internal.appsmith.com/app/cypress-dashboard/rundetails-65890b3c81d7400d08fa9ee5?branch=master&workflowId=9467180088&attempt=1"
target="_blank">Click here!</a>
<!-- end of auto-generated comment: Cypress test results -->
Another attempt at removing QueryDSL. This time, it seems to be working
without very many shenanigans, which throws question on what the hell I
was doing in the previous attempt. 😭
All tests and Sanity Cypress passes on EE at
https://github.com/appsmithorg/appsmith-ee/pull/3805.
Two things happening in this PR.
1. We're removing QueryDSL.
2. We're adding [Errorprone](http://errorprone.info/).
Why both together? Because I can't get the build to work, if I remove
QueryDSL, and not add any other annotation processor. Lombok is refusing
to do its thing, unless there is at least one other annotation
processor, which QueryDSL was filling in for. I wanted to add Errorprone
anyway in a separate task, so didn't investigate further and just added
it here.
But adding Errorprone is not the idea for now. So all lint rules are
disable, and it has no affect on build time either. We'll slowly roll
out some of the checks and lint rules in the coming weeks.
## Description
Mutual TLS, or mTLS for short, is a method for mutual authentication.
mTLS ensures that the parties at each end of a network connection are
who they claim to be by verifying that they both have the correct
private key.
This PR adds support for mTLS for postgres datasource. Unlike the
standard way of storing the certs in disk, we store them in the
database. This has been achieved via the custom implementation of SSL
Factory. The postgres driver support passing the custom ssl factory
while creating connection, which then will be used for handling and
establishing the connection.
#### PR fixes following issue(s)
Fixes#31326
#### Type of change
- New feature (non-breaking change which adds functionality)
## Testing
#### How Has This Been Tested?
- [ ] Manual
#### Test Plan
> Add Testsmith test cases links that relate to this PR
>
>
#### Issues raised during DP testing
> Link issues raised during DP testing for better visiblity and tracking
(copy link from comments dropped on this PR)
>
>
>
## Checklist:
#### Dev activity
- [ ] My code follows the style guidelines of this project
- [ ] I have performed a self-review of my own code
- [ ] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [ ] My changes generate no new warnings
- [ ] I have added tests that prove my fix is effective or that my
feature works
- [ ] New and existing unit tests pass locally with my changes
- [ ] PR is being merged under a feature flag
#### QA activity:
- [ ] [Speedbreak
features](https://github.com/appsmithorg/TestSmith/wiki/Guidelines-for-test-plans#speedbreakers-)
have been covered
- [ ] Test plan covers all impacted features and [areas of
interest](https://github.com/appsmithorg/TestSmith/wiki/Guidelines-for-test-plans#areas-of-interest-)
- [ ] Test plan has been peer reviewed by project stakeholders and other
QA members
- [ ] Manually tested functionality on DP
- [ ] We had an implementation alignment call with stakeholders post QA
Round 2
- [ ] Cypress test cases have been added and approved by SDET/manual QA
- [ ] Added `Test Plan Approved` label after Cypress tests were reviewed
- [ ] Added `Test Plan Approved` label after JUnit tests were reviewed
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Enhanced the Postgres plugin to support SSL certificate validation for
secure database connections. Users can now select between `VERIFY_CA`
and `VERIFY_FULL` modes for SSL verification, ensuring a higher level of
security for database communications.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
QueryDSL will be replaced with Lombok's `FieldNameConstants` here. Why?
1. QueryDSL with JPA on relational databases doesn't lend itself very
well to nested data structures. But with MongoDB, it works quite well.
So we've come to rely on it quite a bit. Since we intend to move towards
a more flat, relation-ed and normalized DB design once we get to
Postgres, dealing with nested data structures should be seen as
temporary.
2. We only use QueryDSL for field name constants, and absolutely nothing
else. QueryDSL is a far more capable and powerful system, and is
overkill for this purpose. Lombok's annotation is exactly tuned for this
purpose and is more concise and easy-to-use.
3. QueryDSL query generation current doesn't work in IntelliJ, but
Lombok's does. So this will free us up from having to run a Maven build
when sometimes switching branches.
**PS**: This PR doesn't remove QueryDSL entirely. Only a part of it.
That'd become a much bigger PR and I'm already uncomfortable with the
size of this PR. Once this is merged, I'll open further PRs until we
completely remove QueryDSL.
**PPS**: QueryDSL is a powerful querying mechanism that we don't use
today. Perhaps once we're comfortable with Postgres in the future, we
will very likely revisit.
## Description
TL;DR: This PR introduces metrics logging using native Spring support
for Micrometer. It includes a docker-compose to set up all the required
parts of this observability stack in the local environment as well.
In order to make use of this stack, please navigate to
`utils/observability` and execute the following command:
```
docker-compose up -d
```
The set up comes bundled with a default Grafana dashboard that can be
accessed at localhost:3001. Please feel free to switch the mapping ports
around in the docker-compose file.
This dashboard currently shows all http requests (sampled at 0.1 by
default), and the server side implementation has introduced some minimal
tracing for the `/api/v1/action/execute` endpoint. This means that you
can use the trace id from http server requests for this endpoint to
delve deeper into the spans exposed in this flow.
In case you would like to send trace information to another service,
please make use of the `APPSMITH_TRACING_ENDPOINT` variable. To override
the default sampling rate in your local (to say, 1), you can set that as
the value for the variable `APPSMITH_SAMPLING_PROBABILITY`.
Fixes#19153
## Type of change
- Chore (housekeeping or task changes that don't impact user perception)
## How Has This Been Tested?
- Manual
### Test Plan
No testing required, only needs regression after merge.
## Checklist:
### Dev activity
- [x] My code follows the style guidelines of this project
- [x] I have performed a self-review of my own code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [ ] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [ ] PR is being merged under a feature flag
---------
Co-authored-by: Sumesh Pradhan <sumesh@appsmith.com>
This upgrade takes care of our move to JDK 17, Spring Boot 3.0.1 and a
few other security upgrades along the way.
Fixes#18993
TODO:
- [x] Check CI changes for Java 17
- [x] Check vulnerability report
- [x] Mongock needs an upgrade
- [x] Add JVM args at all possible places for exposing java.time module
- [x] Add type adapters everywhere / use the same config for type
adapters everywhere
On macOS, when running a MySQL query, we see the following error:
```
java.lang.ClassCastException: class io.netty.channel.kqueue.KQueueEventLoopGroup cannot be cast to class io.netty.channel.EventLoopGroup
```
This is because of a `LinkageError`, caused because these classes exist both in the parent `server` module classpath, as well as in this plugin's classpath separately. To fix this, we removed this dependency from the server module.
But then, this caused a problem with how the `DNSResolver` class was being, well, resolved by the application class loader, vs the plugin class loader. This is a macOS specific problem, and doesn't affect other operating systems.
For this, we add just this package as a dependency to the MySQL plugin.
* Added support for self signed certificate during OAuth2 flows, server side changes
* fix: authentication.useSelfSignedCert key added
* Merging from release
* Fixed issue with dependencies, plus bug with using ssl
* Cypress test attempt 1
* Cypress test attempt 2
* Cypress test attempt 3
* Clean up
* Review comments
* Enabled compression again
Co-authored-by: Aman Agarwal <aman@appsmith.com>
Upgrades vulnerable dependencies in all plugins except for MySQL. That one is still failing and I'll fix it in a separate PR. Issue #14475
Co-authored-by: Nayan <nayan@appsmith.com>
Co-authored-by: Anagh Hegde <anagh@appsmith.com>
- fix Mongo plugin's smart substitution feature to handle quotes around ObjectId in array. e.g. Initial replacement value: ["ObjectId(xyz)"] , final replacement value: [ObjectId(xyz)]
- unrelated: added version number to lombok dependency to stop build failures in IntelliJ.
* core workflows of git - branch, clone, commit, pull, merge, status, log
* Rehydrate file system before merge operation
* Add logic to handle merge conflict for merge and pull flow
* Add defaultBranch to listBranch API
* Add Unit tests for git commands
* Upgrade JGit dependencies to fix the security issue
* Git command tests
* Revert unwanted changes and update delete application flow
* Disable git feature until the FE changes are merged
Co-authored-by: Abhijeet <ABHI.NAGARNAIK@GMAIL.COM>
* POC : null value getting handled correctly
* Refactor code. Tested with and without quotes. This is working! Woohoo!
* Added support for mustache binding to include json objects and arrays
* Added test for smart substitution.
* Added setting to turn on/off smart substitutions in Rest API plugin
* Handling turning on smart json substitution
* Added error handling
* Added config to turn on smart substitution in JSON to make the tests pass now
* Adding a beta tag to the API setting.
* Spelling error resolved.
* Incorporated review comments.
* Catching JSON Parse exception in http call.
- Expect max value of timeout as 60000 ms.
- If value exceeds max value then add error message to the list of invalids. This list is returned to the client in response body.
- Detect integer overflow exception (Number format exception) and override the value to 60000 ms.
* Pushing minor editor form changes to ensure that prepared statement could be turned off.
* Code refactor to do variable substitution in PluginExecutor instead of action service.
* WIP : Prepared Statement handling in psql plugin
* WIP Prepared Statements.
* Working version of prepared statements
* Quote trimming added for post preparing sql statements. Now the unprepared statements and prepared statements do not require edits.
* Fixed existing test cases failing.
* Code formatting.
* Super minor code cleanup.
* Added migration for the existing postgres actions.
* Fixed failing test cases in ActionServiceTest.
* Minor change in the text for turning on and off prepared statements in the postgres query pane.
* Added test cases for prepared statement.
* Some minor comments for code readability
* Moved Prepared Statement setting from Action Configuration to Plugin Specified Templates since this setting does not make sense for all the DB plugins.
* Added function level comments
* Update app/server/appsmith-interfaces/src/main/java/com/appsmith/external/helpers/SqlStringUtils.java
Co-authored-by: Arpit Mohan <mohanarpit@users.noreply.github.com>
* Update app/server/appsmith-interfaces/src/main/java/com/appsmith/external/helpers/SqlStringUtils.java
Co-authored-by: Arpit Mohan <mohanarpit@users.noreply.github.com>
* Incorporated review comments.
* Fixed compile time error.
Co-authored-by: Arpit Mohan <mohanarpit@users.noreply.github.com>
