PromucFlow_constructor/deploy/docker/fs/opt/appsmith/run-java.sh

97 lines
2.6 KiB
Bash
Raw Normal View History

#!/bin/bash
chore: Add postgres dependency for server to startup (#36585) ## Description As in the past we have seen the corruption of postgres DB which is being used for temporal we want to make sure we have a retry mechanism in place if: 1. `APPSMITH_DB_URL` is pointing to postgres url 2. Postgres is waiting in recovery mode As per local testing when the docker container is abruptly shutdown via `docker rm -f {container_name}` or `docker kill {container_name}` or even via docker desktop we end up in state where postgres goes into recovery state. logs: ``` 2024-09-27 08:02:49 backend stdout | SQL State : 57P03 2024-09-27 08:02:49 backend stdout | Error Code : 0 2024-09-27 08:02:49 backend stdout | Message : FATAL: the database system is starting up ``` Currently we have implemented polling mechanism, but we will keep looking for better alternative here if we can opt for. Note: 1. With release dump this is taking ~300sec to get out of that state and start accepting the connections. 2. With the existing implementation without retries server dies down within 60sec. ``` INFO exited: backend (exit status 1; not expected) INFO gave up: backend entered FATAL state, too many start retries too quickly ``` Reference doc: https://www.notion.so/appsmith/Postgres-critical-scenarios-668f49c96aef40308e24c2a8d6b1137c /test Sanity ### :mag: Cypress test results <!-- This is an auto-generated comment: Cypress test results --> > [!TIP] > 🟢 🟢 🟢 All cypress tests have passed! 🎉 🎉 🎉 > Workflow run: <https://github.com/appsmithorg/appsmith/actions/runs/11100944184> > Commit: 9dbbe4b22ba12aa82c385ad0eef3cc3d4876f217 > <a href="https://internal.appsmith.com/app/cypress-dashboard/rundetails-65890b3c81d7400d08fa9ee5?branch=master&workflowId=11100944184&attempt=1" target="_blank">Cypress dashboard</a>. > Tags: `@tag.Sanity` > Spec: > <hr>Mon, 30 Sep 2024 07:26:41 UTC <!-- end of auto-generated comment: Cypress test results --> ## Communication Should the DevRel and Marketing teams inform users about this change? - [ ] Yes - [x] No <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit ## Summary by CodeRabbit - **New Features** - Introduced new functions for enhanced handling of PostgreSQL database connections, including availability checks and parameter extraction. - Added a new utility script for managing PostgreSQL connections. - **Bug Fixes** - Implemented a retry mechanism for PostgreSQL server availability checks to ensure more reliable connections. - **Tests** - Added unit tests to validate the functionality of the PostgreSQL parameter extraction logic. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
2024-09-30 07:27:47 +00:00
# Source the helper script
source pg-utils.sh
set -o errexit
set -o pipefail
set -o nounset
set -o noglob
feat: Make images adaptable to support both Postgres and MongoDB uris (#36424) ## Description PR to make the release tag adaptable to work with both MongoDB and PostgreSQL uris. <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit - **New Features** - Introduced a new script to automate the preparation of server artifacts, improving the build process. - Added SQL files to the indentation configuration for consistent code formatting. - **Improvements** - Enhanced error handling in the Docker build process to ensure essential files are present before execution. - Updated service configuration logic to prevent misconfiguration based on the environment. - Added a new job step in the build workflow to prepare server artifacts after the build process. - Implemented conditional logic in the run script to dynamically adapt to different database configurations. - **Bug Fixes** - Adjusted the initialization process to focus on MongoDB, improving reliability in various environments. <!-- end of auto-generated comment: release notes by coderabbit.ai --> /test Sanity ### :mag: Cypress test results <!-- This is an auto-generated comment: Cypress test results --> > [!TIP] > 🟢 🟢 🟢 All cypress tests have passed! 🎉 🎉 🎉 > Workflow run: <https://github.com/appsmithorg/appsmith/actions/runs/10940528231> > Commit: 32731e8a93a25e5c9456eb89daca2d8bf327c012 > <a href="https://internal.appsmith.com/app/cypress-dashboard/rundetails-65890b3c81d7400d08fa9ee5?branch=master&workflowId=10940528231&attempt=1" target="_blank">Cypress dashboard</a>. > Tags: `@tag.Sanity` > Spec: > <hr>Thu, 19 Sep 2024 12:21:54 UTC <!-- end of auto-generated comment: Cypress test results --> ## Communication Should the DevRel and Marketing teams inform users about this change? - [ ] Yes - [x] No
2024-09-19 16:15:46 +00:00
mode=mongo
if [[ "$APPSMITH_DB_URL" = postgresql://* ]]; then
mode=pg
fi
tlog "Running with $mode."
cd "/opt/appsmith/server/$mode"
declare -a extra_args
proxy_configured=0
match-proxy-url() {
# Examples:
# http://proxy.example.com:8080/
# http://user:pass@proxyhost:123
# http://proxyhost:123
[[ $1 =~ ^http://(([^@:]*):([^@]*)?@)?([^@:]*):([0-9]+)/?$ ]]
proxy_user="${BASH_REMATCH[2]-}"
proxy_pass="${BASH_REMATCH[3]-}"
proxy_host="${BASH_REMATCH[4]-}"
proxy_port="${BASH_REMATCH[5]-}"
[[ -n $proxy_host ]]
}
chore: Add postgres dependency for server to startup (#36585) ## Description As in the past we have seen the corruption of postgres DB which is being used for temporal we want to make sure we have a retry mechanism in place if: 1. `APPSMITH_DB_URL` is pointing to postgres url 2. Postgres is waiting in recovery mode As per local testing when the docker container is abruptly shutdown via `docker rm -f {container_name}` or `docker kill {container_name}` or even via docker desktop we end up in state where postgres goes into recovery state. logs: ``` 2024-09-27 08:02:49 backend stdout | SQL State : 57P03 2024-09-27 08:02:49 backend stdout | Error Code : 0 2024-09-27 08:02:49 backend stdout | Message : FATAL: the database system is starting up ``` Currently we have implemented polling mechanism, but we will keep looking for better alternative here if we can opt for. Note: 1. With release dump this is taking ~300sec to get out of that state and start accepting the connections. 2. With the existing implementation without retries server dies down within 60sec. ``` INFO exited: backend (exit status 1; not expected) INFO gave up: backend entered FATAL state, too many start retries too quickly ``` Reference doc: https://www.notion.so/appsmith/Postgres-critical-scenarios-668f49c96aef40308e24c2a8d6b1137c /test Sanity ### :mag: Cypress test results <!-- This is an auto-generated comment: Cypress test results --> > [!TIP] > 🟢 🟢 🟢 All cypress tests have passed! 🎉 🎉 🎉 > Workflow run: <https://github.com/appsmithorg/appsmith/actions/runs/11100944184> > Commit: 9dbbe4b22ba12aa82c385ad0eef3cc3d4876f217 > <a href="https://internal.appsmith.com/app/cypress-dashboard/rundetails-65890b3c81d7400d08fa9ee5?branch=master&workflowId=11100944184&attempt=1" target="_blank">Cypress dashboard</a>. > Tags: `@tag.Sanity` > Spec: > <hr>Mon, 30 Sep 2024 07:26:41 UTC <!-- end of auto-generated comment: Cypress test results --> ## Communication Should the DevRel and Marketing teams inform users about this change? - [ ] Yes - [x] No <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit ## Summary by CodeRabbit - **New Features** - Introduced new functions for enhanced handling of PostgreSQL database connections, including availability checks and parameter extraction. - Added a new utility script for managing PostgreSQL connections. - **Bug Fixes** - Implemented a retry mechanism for PostgreSQL server availability checks to ensure more reliable connections. - **Tests** - Added unit tests to validate the functionality of the PostgreSQL parameter extraction logic. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
2024-09-30 07:27:47 +00:00
# Extract the database parameters from the APPSMITH_DB_URL and wait for the database to be available
if [[ "$mode" == "pg" ]]; then
extract_postgres_db_params "$APPSMITH_DB_URL"
waitForPostgresAvailability
chore: create appsmith schema for postgres (#36591) ## Description The current state is default schema or public schema. This schema is accessible by default when user connects to the pg database. Hence create `appsmith` schema for Appsmith server to use. This is to avoid anyone accidentally modifying the appsmith data. ## Automation /ok-to-test tags="@tag.Sanity" ### :mag: Cypress test results <!-- This is an auto-generated comment: Cypress test results --> > [!IMPORTANT] > 🟣 🟣 🟣 Your tests are running. > Tests running at: <https://github.com/appsmithorg/appsmith/actions/runs/11111681323> > Commit: 32f91e8d7ce750e4a088996aff4abe6905aa982f > Workflow: `PR Automation test suite` > Tags: `@tag.Sanity` > Spec: `` > <hr>Mon, 30 Sep 2024 18:08:23 UTC <!-- end of auto-generated comment: Cypress test results --> ## Communication Should the DevRel and Marketing teams inform users about this change? - [ ] Yes - [ ] No <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit ## Summary by CodeRabbit - **New Features** - Introduced a script to initialize the PostgreSQL database schema for Appsmith. - Added utilities for managing PostgreSQL database connections, including availability checks and parameter extraction. - Enhanced scripts for managing PostgreSQL connections and initialization. - Improved environment configuration for PostgreSQL database connections, including automatic password generation for local setups. - Updated JDBC URL handling to include schema parameters for PostgreSQL connections. - Added support for proxy configuration in the application setup. - **Bug Fixes** - Improved error handling and connection retry mechanisms for PostgreSQL setup. - **Documentation** - Updated comments and logging for better clarity on database operations. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Co-authored-by: Abhijeet <abhi.nagarnaik@gmail.com>
2024-09-30 18:12:56 +00:00
init_pg_db
chore: Add postgres dependency for server to startup (#36585) ## Description As in the past we have seen the corruption of postgres DB which is being used for temporal we want to make sure we have a retry mechanism in place if: 1. `APPSMITH_DB_URL` is pointing to postgres url 2. Postgres is waiting in recovery mode As per local testing when the docker container is abruptly shutdown via `docker rm -f {container_name}` or `docker kill {container_name}` or even via docker desktop we end up in state where postgres goes into recovery state. logs: ``` 2024-09-27 08:02:49 backend stdout | SQL State : 57P03 2024-09-27 08:02:49 backend stdout | Error Code : 0 2024-09-27 08:02:49 backend stdout | Message : FATAL: the database system is starting up ``` Currently we have implemented polling mechanism, but we will keep looking for better alternative here if we can opt for. Note: 1. With release dump this is taking ~300sec to get out of that state and start accepting the connections. 2. With the existing implementation without retries server dies down within 60sec. ``` INFO exited: backend (exit status 1; not expected) INFO gave up: backend entered FATAL state, too many start retries too quickly ``` Reference doc: https://www.notion.so/appsmith/Postgres-critical-scenarios-668f49c96aef40308e24c2a8d6b1137c /test Sanity ### :mag: Cypress test results <!-- This is an auto-generated comment: Cypress test results --> > [!TIP] > 🟢 🟢 🟢 All cypress tests have passed! 🎉 🎉 🎉 > Workflow run: <https://github.com/appsmithorg/appsmith/actions/runs/11100944184> > Commit: 9dbbe4b22ba12aa82c385ad0eef3cc3d4876f217 > <a href="https://internal.appsmith.com/app/cypress-dashboard/rundetails-65890b3c81d7400d08fa9ee5?branch=master&workflowId=11100944184&attempt=1" target="_blank">Cypress dashboard</a>. > Tags: `@tag.Sanity` > Spec: > <hr>Mon, 30 Sep 2024 07:26:41 UTC <!-- end of auto-generated comment: Cypress test results --> ## Communication Should the DevRel and Marketing teams inform users about this change? - [ ] Yes - [x] No <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit ## Summary by CodeRabbit - **New Features** - Introduced new functions for enhanced handling of PostgreSQL database connections, including availability checks and parameter extraction. - Added a new utility script for managing PostgreSQL connections. - **Bug Fixes** - Implemented a retry mechanism for PostgreSQL server availability checks to ensure more reliable connections. - **Tests** - Added unit tests to validate the functionality of the PostgreSQL parameter extraction logic. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
2024-09-30 07:27:47 +00:00
fi
if match-proxy-url "${HTTP_PROXY-}"; then
extra_args+=(-Dhttp.proxyHost="$proxy_host" -Dhttp.proxyPort="$proxy_port")
if [[ -n $proxy_user ]]; then
extra_args+=(-Dhttp.proxyUser="$proxy_user")
fi
if [[ -n $proxy_pass ]]; then
extra_args+=(-Dhttp.proxyPassword="$proxy_pass")
fi
proxy_configured=1
fi
if match-proxy-url "${HTTPS_PROXY-}"; then
extra_args+=(-Dhttps.proxyHost="$proxy_host" -Dhttps.proxyPort="$proxy_port")
if [[ -n $proxy_user ]]; then
extra_args+=(-Dhttps.proxyUser="$proxy_user")
fi
if [[ -n $proxy_pass ]]; then
extra_args+=(-Dhttps.proxyPassword="$proxy_pass")
fi
proxy_configured=1
fi
2022-07-25 02:12:48 +00:00
if [[ -z "${NO_PROXY-}" ]]; then
# A default for this value is set in entrypoint.sh script.
2022-07-25 02:12:48 +00:00
# If this variable is not set, just set it to empty string.
NO_PROXY=""
fi
if [[ $proxy_configured == 1 ]]; then
extra_args+=(-Djava.net.useSystemProxies=true -Dhttp.nonProxyHosts="${NO_PROXY//,/|}")
fi
if [[ -f "$TMP/java-cacerts-opts" ]]; then
extra_args+=("@$TMP/java-cacerts-opts")
fi
# Wait until RTS started and listens on port 8091
while ! curl --fail --silent localhost:"${APPSMITH_RTS_PORT:-8091}"/rts-api/v1/health-check; do
tlog 'Waiting for RTS to start ...'
sleep 1
done
tlog 'RTS started.'
sh /opt/appsmith/run-starting-page-init.sh &
# Ref -Dlog4j2.formatMsgNoLookups=true https://spring.io/blog/2021/12/10/log4j2-vulnerability-and-spring-boot
exec java ${APPSMITH_JAVA_ARGS:-} ${APPSMITH_JAVA_HEAP_ARG:-} \
--add-opens java.base/java.time=ALL-UNNAMED \
--add-opens java.base/java.nio=ALL-UNNAMED \
-Dserver.port=8080 \
-XX:+ShowCodeDetailsInExceptionMessages \
-Djava.security.egd=file:/dev/./urandom \
-Dlog4j2.formatMsgNoLookups=true \
"${extra_args[@]}" \
-jar server.jar