Majority of the NGINX config is the same, for both HTTP and HTTPS.
Having two separate templates for them is making configuration changes
error-prone, where we often risk forgetting making the same change in
the other file.
This PR merges the two files into one, so the above risk isn't there. It
also makes it easier to experiment with the file while developing, since
we have to make every single change twice during development.
Note: This _will_ cause conflicts in sync, after being merged.
Why are we doing this? This will be a step towards simplifying our
`Dockerfile` with reduced layers and improved caching performance. The
image build time in CI should be faster once this is done.
So far, only calls that go to the Java backend, had the
`X-Content-Type-Options` header in the responses. This PR adds them to
all responses by
1. adding it to NGINX configuration.
2. removing it from Spring security's configuration, so we don't end up
with _two_ `X-Content-Type-Options` headers in the response.
---------
Co-authored-by: Nidhi <nidhi@appsmith.com>
When Appsmith is deployed with HTTPS, all HTTP requests are redirected
to HTTP. But this redirection response, is leaking the NGINX version in
the `Server` header. This PR fixes that.
1. Changing the Maps API Key doesn't need restart anymore.
2. The `isRestartRequired` field in the response of updating env
settings, was being ignored. The client owns the decision of when to
restart (which is correct), so removed this from the server.
3. Write Maps API Key to the database, in the tenant configuration.
4. The Settings page for Maps Ke gets the current value from
`/tenant/current` response, and not `/admin/env`.
5. Removed `APPSMITH_GOOGLE_MAPS_API_KEY` from `/admin/env` response.
6. Tests.
DO NOT MERGE. Please only review/approve. This is expected to break EE
once it goes there, which I intend to solve alongside merging this.
Changing the Maps API Key will update it both in the tenant config in
the database, as well as in the `docker.env` file. This is predominantly
for backwards compatibility, and phased rollout. As part of a separate
PR, we'll have a migration that proactively copies the env variable
value to the database, and comment out the value in the `docker.env`
file. Then we can stop updating the `docker.env` file as well.
## New
## Old
---------
Co-authored-by: Ankita Kinger <ankita@appsmith.com>
## Description
- The default request timeout on nginx is set to 60s. This feat allows
users to configure custom nginx timeout value specified via the
Environment variable `APPSMITH_SERVER_TIMEOUT`
- Solves https://github.com/appsmithorg/appsmith/issues/14535
## Testing
#### How Has This Been Tested?
- [x] Manual
- [ ] Jest
- [ ] Cypress
---------
Co-authored-by: Shrikant Sharat Kandula <shrikant@appsmith.com>
- Removed event listener python script, removed supervisor conf for the
same.
- Added new python script to handle the starting page init.
- Python script will be called via a shell script with a failsafe
removal of the loading page.
- The above shell script will be run as a child of the run java script
Tests:
The child script exits after execution, and does not create zombie
process even when the run java script is called multiple times with
restarts.
Verified exceptions and failsafe manually by introducing syntax errors
and uninstalling requests module.
---------
Co-authored-by: Shrikant Sharat Kandula <shrikant@appsmith.com>
- Added cron task to cleanup log files older than 7 days to run weekly.
- Updated supervisor log rotation config to store only 2 backups for all
processes.
## Description
This PR includes changes for self-hosted instances to utilize an
embedded postgres database for the Template's mockdb.
After this change, by default Appsmith will always run the embedded
Postgres, and connect to it instead of the mockdb cloud instance.
**Solves:**
Issue[#20107](https://github.com/appsmithorg/appsmith/issues/20107)
**Changes:**
- [x] Install Postgresql in the Docker container using Dockerfile :-
_Updated Docker file to install PostgreSQL v13_
- [x] Add provision to stop/disable Postgresql from booting based on an
environment variable :- _To disable embedded postgres, set the **ENV**
var `APPSMITH_ENABLE_EMBEDDED_DB` to `0`; Configured supervisord to
control the lifecycle of the postgres server ( Supervisor uses user
postgres to start the server since root is not allowed)_
- [x] Seed data into the Postgresql DB during first container startup.
This ensures that we don't have to ship the Docker
container with data & bloat it unnecessarily. :- _Curretly, the SQL dump
is baked into the image._
- [x] Persist the Postgresql data in /appsmith-stacks. This ensures that
the data is persisted across version upgrades. :- _The postgres uses the
file system at` /stacks/data/postgres/main` ; note: The file system
owner is user/group postgres_
- [x] Update the mock db endpoint in the product to use the local DB by
default instead of the hosted DB for self-hosted instances. The cloud
instance should still use the hosted DB :- _The embedded postgres
authentication is set to the type `trust`, allowing the existing
template mockdb meta data to be used without any update to the existing
templates mockdb endpoint and credentials_
**Note: There is an additional table `mydb` added to the mockdb for
quickly verifying the db source is the embedded postgres, since the
cloud mockdb does not have that table.**
---------
Co-authored-by: Shrikant Sharat Kandula <shrikant@appsmith.com>
When services within the fat container try to communicate with each
other, like backend to RTS etc., if they use the loopback address of
IPv4, `127.0.0.1`, it works. But if they use the loopback address of
IPv6, `::1`, it fails because the NGINX inside the fat container isn't
set to bind to IPv6.
This PR fixes this.
In EE, we attempt to make connections to Keycloak without setting the
hostname on the `WebClient`. This picks up the hostname of `::1` on
systems with IPv6 as default, and so the communication between backend
and Keycloak fails.
This is affecting users on ECS Fargate, for example.
## Description
We're increasing the default limit of request payload on cloud so that
100 MB files that are base 64 encoded can also be uploaded via Appsmith.
Fixes#20424
## Type of change
- Bug fix (non-breaking change which fixes an issue)
## How Has This Been Tested?
- Manual
### Test Plan
> Add Testsmith test cases links that relate to this PR
### Issues raised during DP testing
> Link issues raised during DP testing for better visiblity and tracking
(copy link from comments dropped on this PR)
## Checklist:
### Dev activity
- [ ] My code follows the style guidelines of this project
- [ ] I have performed a self-review of my own code
- [ ] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [ ] My changes generate no new warnings
- [ ] I have added tests that prove my fix is effective or that my
feature works
- [ ] New and existing unit tests pass locally with my changes
- [ ] PR is being merged under a feature flag
### QA activity:
- [ ] Test plan has been approved by relevant developers
- [ ] Test plan has been peer reviewed by QA
- [ ] Cypress test cases have been added and approved by either SDET or
manual QA
- [ ] Organized project review call with relevant stakeholders after
Round 1/2 of QA
- [ ] Added Test Plan Approved label after reveiwing all Cypress test
Issue: Nginx and RTS used the same env PORT for binding it's service,
while the backend server had the rts port hardcoded on its rts uri.
- Renamed env PORT to APPSMITH_RTS_PORT for starting the rts server.
- Updated nginx config templates to use env `APPSMITH_RTS_PORT`
- Added appsmith.rts.port property in server to use env
APPSMITH_RTS_PORT
- Updated CommonConfig.java rtsBaseDomain to use appsmith.rts.port
---------
Co-authored-by: Shrikant Sharat Kandula <shrikant@appsmith.com>
Fixes#19969.
The previous PR that fixed this was failing with the way we were
creating the initial user on MongoDB. This PR addresses that.
Co-authored-by: Aishwarya UR <aishwarya@appsmith.com>
Fixes: [19207](https://github.com/appsmithorg/appsmith/issues/19207)
**By default on start-up**
```
root@ab348246fff8:/appsmith-stacks/logs# head backend/backend-ab348246fff8.log
Waiting for RTS to start ...
RTS started.
Starting Backend server...
Load environment configuration
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/opt/appsmith/backend/server.jar!/BOOT-INF/lib/logback-classic-1.2.11.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/appsmith/backend/server.jar!/BOOT-INF/lib/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [ch.qos.logback.classic.util.ContextSelectorStaticBinder]
[2022-12-26 11:39:34,962] - Starting ServerApplication vv1.8.14.1 using Java 11.0.17 on ab348246fff8 with PID 1543 (/opt/appsmith/backend/server.jar started by root in /opt/appsmith/backend)
```
**Upon stopping RTS manually**
```
root@ab348246fff8:/appsmith-stacks/logs# supervisorctl stop rts
rts: stopped
root@ab348246fff8:/appsmith-stacks/logs# supervisorctl status
backend RUNNING pid 1497, uptime 0:01:21
cron RUNNING pid 1505, uptime 0:01:21
editor RUNNING pid 1502, uptime 0:01:21
mongodb RUNNING pid 1491, uptime 0:01:21
redis RUNNING pid 1490, uptime 0:01:21
rts STOPPED Dec 26 11:40 AM
stdout RUNNING pid 1489, uptime 0:01:21
```
**Restart Backend service to check if backend service starts**
```
root@ab348246fff8:/appsmith-stacks/logs# supervisorctl restart backend
backend: stopped
backend: started
root@ab348246fff8:/appsmith-stacks/logs# supervisorctl status
backend RUNNING pid 1802, uptime 0:00:12
cron RUNNING pid 1505, uptime 0:01:43
editor RUNNING pid 1502, uptime 0:01:43
mongodb RUNNING pid 1491, uptime 0:01:43
redis RUNNING pid 1490, uptime 0:01:43
rts STOPPED Dec 26 11:40 AM
stdout RUNNING pid 1489, uptime 0:01:43
root@ab348246fff8:/appsmith-stacks/logs# supervisorctl ^C
root@ab348246fff8:/appsmith-stacks/logs# tail -f backend/backend-ab348246fff8.log
Waiting for RTS to start ...
Waiting for RTS to start ...
Waiting for RTS to start ...
Waiting for RTS to start ...
Waiting for RTS to start ...
Waiting for RTS to start ...
Waiting for RTS to start ...
Waiting for RTS to start ...
Waiting for RTS to start ...
Waiting for RTS to start ...
^C
```
**Upon starting RTS the behaviour of backend immediately**
```
root@ab348246fff8:/appsmith-stacks/logs# supervisorctl start rts
rts: started
root@ab348246fff8:/appsmith-stacks/logs# tail -f backend/backend-ab348246fff8.log
Starting Backend server...
Load environment configuration
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/opt/appsmith/backend/server.jar!/BOOT-INF/lib/logback-classic-1.2.11.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/appsmith/backend/server.jar!/BOOT-INF/lib/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [ch.qos.logback.classic.util.ContextSelectorStaticBinder]
[2022-12-26 11:41:33,796] - Starting ServerApplication vv1.8.14.1 using Java 11.0.17 on ab348246fff8 with PID 1968 (/opt/appsmith/backend/server.jar started by root in /opt/appsmith/backend)
[2022-12-26 11:41:33,799] - Running with Spring Boot v2.6.5, Spring v5.3.17
[2022-12-26 11:41:33,803] - No active profile set, falling back to 1 default profile: "default"
[2022-12-26 11:41:34,615] - Multiple Spring Data modules found, entering strict repository configuration mode!
[2022-12-26 11:41:34,616] - Bootstrapping Spring Data Reactive MongoDB repositories in DEFAULT mode.
[2022-12-26 11:41:34,859] - Finished Spring Data repository scanning in 239 ms. Found 55 Reactive MongoDB repository interfaces.
[2022-12-26 11:41:35,220] - Multiple Spring Data modules found, entering strict repository configuration mode!
[2022-12-26 11:41:35,221] - Bootstrapping Spring Data Redis repositories in DEFAULT mode.
```
The control to configure what domains are allowed to embed Appsmith apps is still available, but the default of not allowing anything except for 'self' is changed to allow everything.
While this is convenient, we encourage our users to configure their frame ancestors to limit what domains can embed their Appsmith apps.
Signed-off-by: Shrikant Sharat Kandula <shrikant@appsmith.com>
* Remove optimizely, as it isn't being used anymore from the frontend codebase
* Remove optimizely references form netlify config, vercel config, index.html, docker script, nginx templates and example .env file
