Commit Graph

46 Commits

Author SHA1 Message Date
Nidhi
7e15d8b13d
feat: Server side observability (#19828)
## Description

TL;DR: This PR introduces metrics logging using native Spring support
for Micrometer. It includes a docker-compose to set up all the required
parts of this observability stack in the local environment as well.

In order to make use of this stack, please navigate to
`utils/observability` and execute the following command:
```
docker-compose up -d
```

The set up comes bundled with a default Grafana dashboard that can be
accessed at localhost:3001. Please feel free to switch the mapping ports
around in the docker-compose file.

This dashboard currently shows all http requests (sampled at 0.1 by
default), and the server side implementation has introduced some minimal
tracing for the `/api/v1/action/execute` endpoint. This means that you
can use the trace id from http server requests for this endpoint to
delve deeper into the spans exposed in this flow.

In case you would like to send trace information to another service,
please make use of the `APPSMITH_TRACING_ENDPOINT` variable. To override
the default sampling rate in your local (to say, 1), you can set that as
the value for the variable `APPSMITH_SAMPLING_PROBABILITY`.

Fixes #19153

## Type of change
- Chore (housekeeping or task changes that don't impact user perception)

## How Has This Been Tested?
- Manual

### Test Plan
No testing required, only needs regression after merge.

## Checklist:
### Dev activity
- [x] My code follows the style guidelines of this project
- [x] I have performed a self-review of my own code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [ ] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [ ] PR is being merged under a feature flag

---------

Co-authored-by: Sumesh Pradhan <sumesh@appsmith.com>
2023-02-07 14:26:18 +05:30
Nidhi
dfccd5a058
chore: Upgrade to Spring Boot 3.0.1 (#19044)
This upgrade takes care of our move to JDK 17, Spring Boot 3.0.1 and a
few other security upgrades along the way.

Fixes #18993 

TODO:

- [x] Check CI changes for Java 17
- [x] Check vulnerability report
- [x] Mongock needs an upgrade
- [x] Add JVM args at all possible places for exposing java.time module
- [x] Add type adapters everywhere / use the same config for type
adapters everywhere
2023-01-02 19:40:59 +07:00
Shrikant Sharat Kandula
0989aca511
Upgrade commons-text (#17566) 2022-10-14 19:36:16 +05:30
Nidhi
ca5499c358
chore: Tests pass, server runs w/ analytics (#17205)
* Tests pass, server runs w/ analytics

* Fix okhttp
2022-09-30 11:49:25 +05:30
Nidhi
66f3dfe790
chore: Move to Jupiter JUnit5 (#17083)
* Upgrading to Jupiter tests for everything but plugins

* Whoops, too soon

* Fixed failing tests

* Imports fixed

* Removed PowerMock dep and JUnit4 from interfaces

* Woohoo! That's testcontainers

* Added maven enforcer to disallow junit4

* Update app/server/appsmith-server/src/test/java/com/appsmith/server/services/GitServiceTest.java

Co-authored-by: Shrikant Sharat Kandula <shrikant@appsmith.com>

* Update app/server/appsmith-server/src/test/java/com/appsmith/server/services/UserWorkspaceServiceTest.java

Co-authored-by: Shrikant Sharat Kandula <shrikant@appsmith.com>

* Update app/server/appsmith-server/src/test/java/com/appsmith/server/services/ce/ActionServiceCE_Test.java

Co-authored-by: Shrikant Sharat Kandula <shrikant@appsmith.com>

* Update app/server/appsmith-server/src/test/java/com/appsmith/server/solutions/CreateDBTablePageSolutionTests.java

Co-authored-by: Shrikant Sharat Kandula <shrikant@appsmith.com>

* Weird formatting and version upgrades

Co-authored-by: Shrikant Sharat Kandula <shrikant@appsmith.com>
2022-09-29 08:54:07 +05:30
Shrikant Sharat Kandula
98776e9e94
Fix h2 version mismatch (#16845) 2022-09-19 08:04:30 +05:30
Sumit Kumar
0cbe81dfde
feat: merge GraphQL plugin to release (#16554)
* merge GraphQL plugin to release
2022-09-09 21:29:47 +05:30
Shrikant Sharat Kandula
06c8210a3a
fix: Fix LinkageError with macOS DNSResolver class (#16581)
On macOS, when running a MySQL query, we see the following error:

```
java.lang.ClassCastException: class io.netty.channel.kqueue.KQueueEventLoopGroup cannot be cast to class io.netty.channel.EventLoopGroup
```

This is because of a `LinkageError`, caused because these classes exist both in the parent `server` module classpath, as well as in this plugin's classpath separately. To fix this, we removed this dependency from the server module.

But then, this caused a problem with how the `DNSResolver` class was being, well, resolved by the application class loader, vs the plugin class loader. This is a macOS specific problem, and doesn't affect other operating systems.

For this, we add just this package as a dependency to the MySQL plugin.
2022-09-06 22:00:53 +05:30
Shrikant Sharat Kandula
8a0838505a
All outgoing requests should go through the configured proxy (#14427)
Signed-off-by: Shrikant Sharat Kandula <shrikant@appsmith.com>
Co-authored-by: Aishwarya UR <aishwarya@appsmith.com>
2022-09-01 11:16:45 +05:30
Nidhi
50172ecd64
fix: Added support for self signed certificate during OAuth2 flows (#14719)
* Added support for self signed certificate during OAuth2 flows, server side changes

* fix: authentication.useSelfSignedCert key added

* Merging from release

* Fixed issue with dependencies, plus bug with using ssl

* Cypress test attempt 1

* Cypress test attempt 2

* Cypress test attempt 3

* Clean up

* Review comments

* Enabled compression again

Co-authored-by: Aman Agarwal <aman@appsmith.com>
2022-07-21 16:10:36 +05:30
Shrikant Sharat Kandula
bba78afc3b
chore: Upgrade dependencies reported by Dependabot (#13735)
Upgrades vulnerable dependencies in all plugins except for MySQL. That one is still failing and I'll fix it in a separate PR. Issue #14475

Co-authored-by: Nayan <nayan@appsmith.com>
Co-authored-by: Anagh Hegde <anagh@appsmith.com>
2022-07-19 10:23:27 +05:30
subratadeypappu
531f5f89d5
fix: Upgrade Lombok package version to 1.18.22 to avoid build error (#14080) (#14156)
Update lombok packages to version 1.18.22
2022-06-01 10:29:42 +00:00
dependabot[bot]
113c96dc54
chore: bump h2 from 2.0.206 to 2.1.210 in /app/server/appsmith-interfaces (#10554)
Bumps [h2](https://github.com/h2database/h2database) from 2.0.206 to 2.1.210.
- [Release notes](https://github.com/h2database/h2database/releases)
- [Commits](https://github.com/h2database/h2database/compare/version-2.0.206...version-2.1.210)

---
updated-dependencies:
- dependency-name: com.h2database:h2
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-01-25 19:38:36 +05:30
dependabot[bot]
7d64d7fa53
chore: bump h2 from 2.0.202 to 2.0.206 in /app/server/appsmith-interfaces (#10414)
Bumps [h2](https://github.com/h2database/h2database) from 2.0.202 to 2.0.206.
- [Release notes](https://github.com/h2database/h2database/releases)
- [Commits](https://github.com/h2database/h2database/compare/version-2.0.202...version-2.0.206)

---
updated-dependencies:
- dependency-name: com.h2database:h2
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-01-15 04:13:05 +05:30
Snyk bot
65b5b61840
chore: [Snyk] Security upgrade com.h2database:h2 from 1.4.200 to 2.0.202 (#9500) 2022-01-15 03:37:21 +05:30
Sumit Kumar
c858edf726
fix Mongo smart substitution for quotes around ObjectId (#9856)
- fix Mongo plugin's smart substitution feature to handle quotes around ObjectId in array. e.g. Initial replacement value: ["ObjectId(xyz)"] , final replacement value: [ObjectId(xyz)]
- unrelated: added version number to lombok dependency to stop build failures in IntelliJ.
2021-12-21 18:47:19 +05:30
Nayan
28c181b59c
feat: [Feature] Upgrade spring boot version to 2.5.5 (#8932)
Upgrades spring boot version to 2.5.5
2021-12-14 16:22:37 +06:00
Anagh Hegde
6faaa33936
feat: Git version control functionality with connect to remote, commit, push flows (#8403)
* core workflows of git - branch, clone, commit, pull, merge, status, log

* Rehydrate file system before merge operation

* Add logic to handle merge conflict for merge and pull flow

* Add defaultBranch to listBranch API

* Add Unit tests for git commands

* Upgrade JGit dependencies to fix the security issue

* Git command tests

* Revert unwanted changes and update delete application flow

* Disable git feature until the FE changes are merged

Co-authored-by: Abhijeet <ABHI.NAGARNAIK@GMAIL.COM>
2021-10-20 09:47:34 +00:00
Nidhi
c3f4cdaa15
feat: Introducing SaaS integrations as a plugin (#7560)
* WIP client side changes

* Saas execution flow + scheduled import of remote plugins
2021-09-21 17:35:29 +05:30
Trisha Anand
659d7c3866
feat: Where condition helper library using H2 in memory database (#7592) 2021-09-19 20:33:47 +05:30
Snyk bot
8428990cee
fix: app/server/appsmith-interfaces/pom.xml to reduce vulnerabilities (#6063)
The following vulnerabilities are fixed with an upgrade:
- https://snyk.io/vuln/SNYK-JAVA-COMGOOGLEGUAVA-1015415
- https://snyk.io/vuln/SNYK-JAVA-COMGOOGLEGUAVA-32236
2021-07-26 13:30:27 +05:30
dependabot[bot]
aafefe8ce9
Bump hibernate-validator in /app/server/appsmith-interfaces (#4937)
Bumps [hibernate-validator](https://github.com/hibernate/hibernate-validator) from 6.0.18.Final to 6.0.20.Final.
- [Release notes](https://github.com/hibernate/hibernate-validator/releases)
- [Changelog](https://github.com/hibernate/hibernate-validator/blob/6.0.20.Final/changelog.txt)
- [Commits](https://github.com/hibernate/hibernate-validator/compare/6.0.18.Final...6.0.20.Final)

---
updated-dependencies:
- dependency-name: org.hibernate.validator:hibernate-validator
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-07-08 18:12:58 +05:30
abhishek nayak
37708a4128
Merge pull request #5231 from appsmithorg/snyk-fix-45d4107a9d315a1ce1ee74338cb35c92
[Snyk] Fix for 26 vulnerabilities
2021-06-18 20:42:01 +05:30
Snyk bot
5c17d962a0
[Snyk] Security upgrade net.minidev:json-smart from 2.3 to 2.4.7 (#5225)
* fix: app/server/appsmith-interfaces/pom.xml to reduce vulnerabilities

The following vulnerabilities are fixed with an upgrade:
- https://snyk.io/vuln/SNYK-JAVA-NETMINIDEV-1078499
- https://snyk.io/vuln/SNYK-JAVA-NETMINIDEV-1298655

* Updating the json-smart dependency to the latest one

Co-authored-by: Arpit Mohan <mohanarpit@users.noreply.github.com>
2021-06-18 10:54:34 +05:30
snyk-bot
e8d731a21a
fix: app/server/appsmith-interfaces/pom.xml & app/server/appsmith-server/pom.xml to reduce vulnerabilities
The following vulnerabilities are fixed with an upgrade:
- https://snyk.io/vuln/SNYK-JAVA-COMFASTERXMLJACKSONCORE-1052449
- https://snyk.io/vuln/SNYK-JAVA-COMFASTERXMLJACKSONCORE-1052450
- https://snyk.io/vuln/SNYK-JAVA-COMFASTERXMLJACKSONCORE-1054588
- https://snyk.io/vuln/SNYK-JAVA-COMFASTERXMLJACKSONCORE-1056414
- https://snyk.io/vuln/SNYK-JAVA-COMFASTERXMLJACKSONCORE-1056416
- https://snyk.io/vuln/SNYK-JAVA-COMFASTERXMLJACKSONCORE-1056417
- https://snyk.io/vuln/SNYK-JAVA-COMFASTERXMLJACKSONCORE-1056418
- https://snyk.io/vuln/SNYK-JAVA-COMFASTERXMLJACKSONCORE-1056419
- https://snyk.io/vuln/SNYK-JAVA-COMFASTERXMLJACKSONCORE-1056420
- https://snyk.io/vuln/SNYK-JAVA-COMFASTERXMLJACKSONCORE-1056421
- https://snyk.io/vuln/SNYK-JAVA-COMFASTERXMLJACKSONCORE-1056424
- https://snyk.io/vuln/SNYK-JAVA-COMFASTERXMLJACKSONCORE-1056425
- https://snyk.io/vuln/SNYK-JAVA-COMFASTERXMLJACKSONCORE-1056426
- https://snyk.io/vuln/SNYK-JAVA-COMFASTERXMLJACKSONCORE-1056427
- https://snyk.io/vuln/SNYK-JAVA-COMMONSBEANUTILS-460111
- https://snyk.io/vuln/SNYK-JAVA-IONETTY-1020439
- https://snyk.io/vuln/SNYK-JAVA-IONETTY-1070799
- https://snyk.io/vuln/SNYK-JAVA-IONETTY-1082238
- https://snyk.io/vuln/SNYK-JAVA-IONETTY-1083991
- https://snyk.io/vuln/SNYK-JAVA-IONETTY-1089809
- https://snyk.io/vuln/SNYK-JAVA-IOPROJECTREACTORNETTY-559154
- https://snyk.io/vuln/SNYK-JAVA-IOPROJECTREACTORNETTY-559155
- https://snyk.io/vuln/SNYK-JAVA-ORGSPRINGFRAMEWORK-1009832
- https://snyk.io/vuln/SNYK-JAVA-ORGSPRINGFRAMEWORK-1296829
- https://snyk.io/vuln/SNYK-JAVA-ORGSPRINGFRAMEWORKSECURITY-1290497
- https://snyk.io/vuln/SNYK-JAVA-ORGYAML-537645
2021-06-17 13:27:03 +00:00
dependabot[bot]
3573ea3913
Bump commons-io from 2.6 to 2.7 in /app/server/appsmith-interfaces (#4172)
Bumps commons-io from 2.6 to 2.7.

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-05-09 09:49:43 +05:30
Trisha Anand
9f63bfafd2
Adding JSON intelligent substitution for Rest Api plugin (#3474)
* POC : null value getting handled correctly

* Refactor code. Tested with and without quotes. This is working! Woohoo!

* Added support for mustache binding to include json objects and arrays

* Added test for smart substitution.

* Added setting to turn on/off smart substitutions in Rest API plugin

* Handling turning on smart json substitution

* Added error handling

* Added config to turn on smart substitution in JSON to make the tests pass now

* Adding a beta tag to the API setting.

* Spelling error resolved.

* Incorporated review comments.

* Catching JSON Parse exception in http call.
2021-03-11 14:10:00 +05:30
Sumit Kumar
800d305dab
Gracefully handle the timeout overflow / out of expected range errors (#3411)
- Expect max value of timeout as 60000 ms.
- If value exceeds max value then add error message to the list of invalids. This list is returned to the client in response body.
- Detect integer overflow exception (Number format exception) and override the value to 60000 ms.
2021-03-05 17:09:29 +05:30
Trisha Anand
e5574c1945
Support Prepared Statements in Postgres (#2967)
* Pushing minor editor form changes to ensure that prepared statement could be turned off.

* Code refactor to do variable substitution in PluginExecutor instead of action service.

* WIP : Prepared Statement handling in psql plugin

* WIP Prepared Statements.

* Working version of prepared statements

* Quote trimming added for post preparing sql statements. Now the unprepared statements and prepared statements do not require edits.

* Fixed existing test cases failing.

* Code formatting.

* Super minor code cleanup.

* Added migration for the existing postgres actions.

* Fixed failing test cases in ActionServiceTest.

* Minor change in the text for turning on and off prepared statements in the postgres query pane.

* Added test cases for prepared statement.

* Some minor comments for code readability

* Moved Prepared Statement setting from Action Configuration to Plugin Specified Templates since this setting does not make sense for all the DB plugins.

* Added function level comments

* Update app/server/appsmith-interfaces/src/main/java/com/appsmith/external/helpers/SqlStringUtils.java

Co-authored-by: Arpit Mohan <mohanarpit@users.noreply.github.com>

* Update app/server/appsmith-interfaces/src/main/java/com/appsmith/external/helpers/SqlStringUtils.java

Co-authored-by: Arpit Mohan <mohanarpit@users.noreply.github.com>

* Incorporated review comments.

* Fixed compile time error.

Co-authored-by: Arpit Mohan <mohanarpit@users.noreply.github.com>
2021-02-18 18:33:27 +05:30
dependabot[bot]
e781ceb30a
Bump jackson-databind in /app/server/appsmith-interfaces (#2652)
Bumps [jackson-databind](https://github.com/FasterXML/jackson) from 2.9.10.5 to 2.9.10.7.
- [Release notes](https://github.com/FasterXML/jackson/releases)
- [Commits](https://github.com/FasterXML/jackson/commits)

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-01-21 10:48:01 +05:30
Trisha Anand
2062c60ca3
Moved all the scheduling of the plugin executions to bounded elastic instead of elastic. (#1931) 2020-11-26 17:50:43 +05:30
dependabot[bot]
668c8e9c1c
Bump junit from 4.11 to 4.13.1 in /app/server/appsmith-interfaces (#1161)
Bumps [junit](https://github.com/junit-team/junit4) from 4.11 to 4.13.1.
- [Release notes](https://github.com/junit-team/junit4/releases)
- [Changelog](https://github.com/junit-team/junit4/blob/main/doc/ReleaseNotes4.11.md)
- [Commits](https://github.com/junit-team/junit4/compare/r4.11...r4.13.1)

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2020-10-13 14:33:50 +05:30
Shrikant Sharat Kandula
13ddcc0cc9
Upgrade pf4j & test containers and fix plugin tests (#1156) 2020-10-13 13:42:14 +05:30
Trisha Anand
8734067cad
Resolving the dependabot vulnerabilities. (#69) 2020-07-09 21:12:16 +05:30
Trisha Anand
6be0b7ae23 Merge branch 'release' into feature/acl-spring-object
# Conflicts:
#	appsmith-server/pom.xml
#	appsmith-server/src/main/java/com/appsmith/server/migrations/DatabaseChangelog.java
#	appsmith-server/src/main/java/com/appsmith/server/repositories/BaseRepositoryImpl.java
#	appsmith-server/src/main/java/com/appsmith/server/services/LayoutActionServiceImpl.java
#	appsmith-server/src/test/java/com/appsmith/server/services/OrganizationServiceTest.java
2020-04-20 18:13:55 +05:30
Shrikant Kandula
42d0060ecd Upgrade pf4j-spring to 0.6.0 so we can consistently use pf4j 3.2.0. 2020-04-09 19:55:58 +05:30
Shrikant Kandula
ff6b0ce6bc Increase restapi client buffer size from 256KB to 10MB. 2020-04-07 11:20:46 +00:00
Trisha Anand
5911234276 To ensure that special characters like "\n", "\t", etc are preserved in the request json body, convert the json string into object. 2020-04-03 14:27:10 +00:00
Arpit Mohan
0914acdca6 Adding default principal for anonymous Users in SecurityConfig
Also, hard-coding the document fields in BaseRepositoryImpl criteria queries.
2020-03-04 18:57:46 +05:30
Arpit Mohan
7019bae082 Adding policy to the document object instead of the user
The login has broken and all users are being treated as un-authenticated users
2020-03-04 18:57:46 +05:30
Arpit Mohan
3a1673dfca Adding QueryDsl to the appsmith-server and appsmith-interfaces modules
This will generate the annotated classes so that we can refer to the fields easily and scalably.
2020-03-04 18:57:46 +05:30
Arpit Mohan
3a9f344a51 Removing plugins from appsmith-interfaces module 2019-10-17 09:25:58 +05:30
Trisha Anand
8658df95a9 All action executions now return object of type ActionExecutionResult. RestApiPlugin returns the same object which contains statusCode, headers and body. 2019-09-30 18:17:35 +00:00
Trisha Anand
bec8b2e0a1 Adding the docker-compose up command to the build
This should ideally run the mongo-seed container to restore the DB before each run
2019-09-24 12:14:41 +00:00
Arpit Mohan
0edc0f6363 Adding the executeAction API in the ActionController.
Also changed the interface for the PluginExecutor to include the resourceConfig and ActionConfig as well.
2019-09-17 12:18:23 +00:00
Arpit Mohan
b3c987a9f9 Convert the project into a multi-module structure
We are moving to a multi-module structure so that different parts of the codebase can be exposed to the public while others can remain private. Using pf4j for plugin framework.

Also adding a build script `build.sh` which compiles the code and creates the `dist` folder for distribution purposes. Now we can build the code via

```
$ ./build.sh -DskipTests
```
2019-09-16 11:21:57 +00:00