PromucFlow_constructor/deploy/docker/fs/opt/appsmith/pg-utils.sh

200 lines
8.1 KiB
Bash
Raw Normal View History

chore: Add postgres dependency for server to startup (#36585) ## Description As in the past we have seen the corruption of postgres DB which is being used for temporal we want to make sure we have a retry mechanism in place if: 1. `APPSMITH_DB_URL` is pointing to postgres url 2. Postgres is waiting in recovery mode As per local testing when the docker container is abruptly shutdown via `docker rm -f {container_name}` or `docker kill {container_name}` or even via docker desktop we end up in state where postgres goes into recovery state. logs: ``` 2024-09-27 08:02:49 backend stdout | SQL State : 57P03 2024-09-27 08:02:49 backend stdout | Error Code : 0 2024-09-27 08:02:49 backend stdout | Message : FATAL: the database system is starting up ``` Currently we have implemented polling mechanism, but we will keep looking for better alternative here if we can opt for. Note: 1. With release dump this is taking ~300sec to get out of that state and start accepting the connections. 2. With the existing implementation without retries server dies down within 60sec. ``` INFO exited: backend (exit status 1; not expected) INFO gave up: backend entered FATAL state, too many start retries too quickly ``` Reference doc: https://www.notion.so/appsmith/Postgres-critical-scenarios-668f49c96aef40308e24c2a8d6b1137c /test Sanity ### :mag: Cypress test results <!-- This is an auto-generated comment: Cypress test results --> > [!TIP] > 🟢 🟢 🟢 All cypress tests have passed! 🎉 🎉 🎉 > Workflow run: <https://github.com/appsmithorg/appsmith/actions/runs/11100944184> > Commit: 9dbbe4b22ba12aa82c385ad0eef3cc3d4876f217 > <a href="https://internal.appsmith.com/app/cypress-dashboard/rundetails-65890b3c81d7400d08fa9ee5?branch=master&workflowId=11100944184&attempt=1" target="_blank">Cypress dashboard</a>. > Tags: `@tag.Sanity` > Spec: > <hr>Mon, 30 Sep 2024 07:26:41 UTC <!-- end of auto-generated comment: Cypress test results --> ## Communication Should the DevRel and Marketing teams inform users about this change? - [ ] Yes - [x] No <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit ## Summary by CodeRabbit - **New Features** - Introduced new functions for enhanced handling of PostgreSQL database connections, including availability checks and parameter extraction. - Added a new utility script for managing PostgreSQL connections. - **Bug Fixes** - Implemented a retry mechanism for PostgreSQL server availability checks to ensure more reliable connections. - **Tests** - Added unit tests to validate the functionality of the PostgreSQL parameter extraction logic. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
2024-09-30 07:27:47 +00:00
#!/bin/bash
chore: Add grants for appsmith user for embedded postgres (#36664) ## Description PR to add the necessary grants to `appsmith` user when user opts for Postgres embedded DB. fixes https://github.com/appsmithorg/appsmith/issues/36661 ## Automation /test Sanity ### :mag: Cypress test results <!-- This is an auto-generated comment: Cypress test results --> > [!TIP] > 🟢 🟢 🟢 All cypress tests have passed! 🎉 🎉 🎉 > Workflow run: <https://github.com/appsmithorg/appsmith/actions/runs/11155064003> > Commit: 1fb82e52a3f05da1b9bb4082cf871a62d3554d78 > <a href="https://internal.appsmith.com/app/cypress-dashboard/rundetails-65890b3c81d7400d08fa9ee5?branch=master&workflowId=11155064003&attempt=2" target="_blank">Cypress dashboard</a>. > Tags: `@tag.Sanity` > Spec: > <hr>Thu, 03 Oct 2024 04:22:54 UTC <!-- end of auto-generated comment: Cypress test results --> ## Communication Should the DevRel and Marketing teams inform users about this change? - [ ] Yes - [x] No <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit - **New Features** - Introduced default values for PostgreSQL database connection parameters. - Added a new function to manage user permissions on database schemas. - **Improvements** - Enhanced the existing database initialization process to include permission granting after schema verification. - Updated documentation within the script for better clarity on new functionalities. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
2024-10-04 04:47:17 +00:00
# default values
DB_USER="appsmith"
DB_HOST="127.0.0.1"
DB_PORT="5432"
DB_SCHEMA="appsmith"
DB_NAME="appsmith"
POSTGRES_ADMIN_USER="postgres"
POSTGRES_DB_PATH="/appsmith-stacks/data/postgres/main"
chore: Add grants for appsmith user for embedded postgres (#36664) ## Description PR to add the necessary grants to `appsmith` user when user opts for Postgres embedded DB. fixes https://github.com/appsmithorg/appsmith/issues/36661 ## Automation /test Sanity ### :mag: Cypress test results <!-- This is an auto-generated comment: Cypress test results --> > [!TIP] > 🟢 🟢 🟢 All cypress tests have passed! 🎉 🎉 🎉 > Workflow run: <https://github.com/appsmithorg/appsmith/actions/runs/11155064003> > Commit: 1fb82e52a3f05da1b9bb4082cf871a62d3554d78 > <a href="https://internal.appsmith.com/app/cypress-dashboard/rundetails-65890b3c81d7400d08fa9ee5?branch=master&workflowId=11155064003&attempt=2" target="_blank">Cypress dashboard</a>. > Tags: `@tag.Sanity` > Spec: > <hr>Thu, 03 Oct 2024 04:22:54 UTC <!-- end of auto-generated comment: Cypress test results --> ## Communication Should the DevRel and Marketing teams inform users about this change? - [ ] Yes - [x] No <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit - **New Features** - Introduced default values for PostgreSQL database connection parameters. - Added a new function to manage user permissions on database schemas. - **Improvements** - Enhanced the existing database initialization process to include permission granting after schema verification. - Updated documentation within the script for better clarity on new functionalities. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
2024-10-04 04:47:17 +00:00
chore: Add postgres dependency for server to startup (#36585) ## Description As in the past we have seen the corruption of postgres DB which is being used for temporal we want to make sure we have a retry mechanism in place if: 1. `APPSMITH_DB_URL` is pointing to postgres url 2. Postgres is waiting in recovery mode As per local testing when the docker container is abruptly shutdown via `docker rm -f {container_name}` or `docker kill {container_name}` or even via docker desktop we end up in state where postgres goes into recovery state. logs: ``` 2024-09-27 08:02:49 backend stdout | SQL State : 57P03 2024-09-27 08:02:49 backend stdout | Error Code : 0 2024-09-27 08:02:49 backend stdout | Message : FATAL: the database system is starting up ``` Currently we have implemented polling mechanism, but we will keep looking for better alternative here if we can opt for. Note: 1. With release dump this is taking ~300sec to get out of that state and start accepting the connections. 2. With the existing implementation without retries server dies down within 60sec. ``` INFO exited: backend (exit status 1; not expected) INFO gave up: backend entered FATAL state, too many start retries too quickly ``` Reference doc: https://www.notion.so/appsmith/Postgres-critical-scenarios-668f49c96aef40308e24c2a8d6b1137c /test Sanity ### :mag: Cypress test results <!-- This is an auto-generated comment: Cypress test results --> > [!TIP] > 🟢 🟢 🟢 All cypress tests have passed! 🎉 🎉 🎉 > Workflow run: <https://github.com/appsmithorg/appsmith/actions/runs/11100944184> > Commit: 9dbbe4b22ba12aa82c385ad0eef3cc3d4876f217 > <a href="https://internal.appsmith.com/app/cypress-dashboard/rundetails-65890b3c81d7400d08fa9ee5?branch=master&workflowId=11100944184&attempt=1" target="_blank">Cypress dashboard</a>. > Tags: `@tag.Sanity` > Spec: > <hr>Mon, 30 Sep 2024 07:26:41 UTC <!-- end of auto-generated comment: Cypress test results --> ## Communication Should the DevRel and Marketing teams inform users about this change? - [ ] Yes - [x] No <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit ## Summary by CodeRabbit - **New Features** - Introduced new functions for enhanced handling of PostgreSQL database connections, including availability checks and parameter extraction. - Added a new utility script for managing PostgreSQL connections. - **Bug Fixes** - Implemented a retry mechanism for PostgreSQL server availability checks to ensure more reliable connections. - **Tests** - Added unit tests to validate the functionality of the PostgreSQL parameter extraction logic. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
2024-09-30 07:27:47 +00:00
waitForPostgresAvailability() {
if [ -z "$PG_DB_HOST" ]; then
tlog "PostgreSQL host name is empty. Check env variables. Error. Exiting java setup"
exit 2
else
MAX_RETRIES=50
RETRYSECONDS=10
retry_count=0
fix: Server startup issue for external postgres (#38019) ## Description We introduced the auth for embedded Postgres, but in pg-utils we missed on updating the pg_isready condition to resort to db host in case of external postgres url. This caused the pg_isready to never pass when user configure external postgres url as a persistence DB in Appsmith. This PR checks if the local postgres url is configured and accordingly modifies the pg_isready command to target the embedded host vs external host. Why it did not get caught in our testing? We are still investigating this but in local for some reason even if we opt for external Postgres DB embedded Postgres process is also initiating which runs the pg_isready against the embedded Postgres and startup goes through. Fixes #`Issue Number` _or_ Fixes `Issue URL` > [!WARNING] > _If no issue exists, please create an issue first, and check with the maintainers if the issue is valid._ ## Automation /ok-to-test tags="" ### :mag: Cypress test results <!-- This is an auto-generated comment: Cypress test results --> > [!CAUTION] > If you modify the content in this section, you are likely to disrupt the CI result for your PR. <!-- end of auto-generated comment: Cypress test results --> ## Communication Should the DevRel and Marketing teams inform users about this change? - [ ] Yes - [ ] No <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit - **New Features** - Enhanced PostgreSQL connection handling with dynamic checks for local or remote instances. - Improved flexibility in connection strategy for better database accessibility. - **Bug Fixes** - Updated connection commands to ensure proper usage of the new host proxy variable. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
2024-12-06 11:08:12 +00:00
local host_proxy
if [[ "$PG_DB_HOST" == "localhost" || "$PG_DB_HOST" == "127.0.0.1" ]]; then
host_proxy=$(get_unix_socket_directory)
else
host_proxy=$PG_DB_HOST
fi
chore: Add postgres dependency for server to startup (#36585) ## Description As in the past we have seen the corruption of postgres DB which is being used for temporal we want to make sure we have a retry mechanism in place if: 1. `APPSMITH_DB_URL` is pointing to postgres url 2. Postgres is waiting in recovery mode As per local testing when the docker container is abruptly shutdown via `docker rm -f {container_name}` or `docker kill {container_name}` or even via docker desktop we end up in state where postgres goes into recovery state. logs: ``` 2024-09-27 08:02:49 backend stdout | SQL State : 57P03 2024-09-27 08:02:49 backend stdout | Error Code : 0 2024-09-27 08:02:49 backend stdout | Message : FATAL: the database system is starting up ``` Currently we have implemented polling mechanism, but we will keep looking for better alternative here if we can opt for. Note: 1. With release dump this is taking ~300sec to get out of that state and start accepting the connections. 2. With the existing implementation without retries server dies down within 60sec. ``` INFO exited: backend (exit status 1; not expected) INFO gave up: backend entered FATAL state, too many start retries too quickly ``` Reference doc: https://www.notion.so/appsmith/Postgres-critical-scenarios-668f49c96aef40308e24c2a8d6b1137c /test Sanity ### :mag: Cypress test results <!-- This is an auto-generated comment: Cypress test results --> > [!TIP] > 🟢 🟢 🟢 All cypress tests have passed! 🎉 🎉 🎉 > Workflow run: <https://github.com/appsmithorg/appsmith/actions/runs/11100944184> > Commit: 9dbbe4b22ba12aa82c385ad0eef3cc3d4876f217 > <a href="https://internal.appsmith.com/app/cypress-dashboard/rundetails-65890b3c81d7400d08fa9ee5?branch=master&workflowId=11100944184&attempt=1" target="_blank">Cypress dashboard</a>. > Tags: `@tag.Sanity` > Spec: > <hr>Mon, 30 Sep 2024 07:26:41 UTC <!-- end of auto-generated comment: Cypress test results --> ## Communication Should the DevRel and Marketing teams inform users about this change? - [ ] Yes - [x] No <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit ## Summary by CodeRabbit - **New Features** - Introduced new functions for enhanced handling of PostgreSQL database connections, including availability checks and parameter extraction. - Added a new utility script for managing PostgreSQL connections. - **Bug Fixes** - Implemented a retry mechanism for PostgreSQL server availability checks to ensure more reliable connections. - **Tests** - Added unit tests to validate the functionality of the PostgreSQL parameter extraction logic. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
2024-09-30 07:27:47 +00:00
while true; do
fix: Server startup issue for external postgres (#38019) ## Description We introduced the auth for embedded Postgres, but in pg-utils we missed on updating the pg_isready condition to resort to db host in case of external postgres url. This caused the pg_isready to never pass when user configure external postgres url as a persistence DB in Appsmith. This PR checks if the local postgres url is configured and accordingly modifies the pg_isready command to target the embedded host vs external host. Why it did not get caught in our testing? We are still investigating this but in local for some reason even if we opt for external Postgres DB embedded Postgres process is also initiating which runs the pg_isready against the embedded Postgres and startup goes through. Fixes #`Issue Number` _or_ Fixes `Issue URL` > [!WARNING] > _If no issue exists, please create an issue first, and check with the maintainers if the issue is valid._ ## Automation /ok-to-test tags="" ### :mag: Cypress test results <!-- This is an auto-generated comment: Cypress test results --> > [!CAUTION] > If you modify the content in this section, you are likely to disrupt the CI result for your PR. <!-- end of auto-generated comment: Cypress test results --> ## Communication Should the DevRel and Marketing teams inform users about this change? - [ ] Yes - [ ] No <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit - **New Features** - Enhanced PostgreSQL connection handling with dynamic checks for local or remote instances. - Improved flexibility in connection strategy for better database accessibility. - **Bug Fixes** - Updated connection commands to ensure proper usage of the new host proxy variable. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
2024-12-06 11:08:12 +00:00
su postgres -c "pg_isready -h $host_proxy -p '${PG_DB_PORT}'"
chore: Add postgres dependency for server to startup (#36585) ## Description As in the past we have seen the corruption of postgres DB which is being used for temporal we want to make sure we have a retry mechanism in place if: 1. `APPSMITH_DB_URL` is pointing to postgres url 2. Postgres is waiting in recovery mode As per local testing when the docker container is abruptly shutdown via `docker rm -f {container_name}` or `docker kill {container_name}` or even via docker desktop we end up in state where postgres goes into recovery state. logs: ``` 2024-09-27 08:02:49 backend stdout | SQL State : 57P03 2024-09-27 08:02:49 backend stdout | Error Code : 0 2024-09-27 08:02:49 backend stdout | Message : FATAL: the database system is starting up ``` Currently we have implemented polling mechanism, but we will keep looking for better alternative here if we can opt for. Note: 1. With release dump this is taking ~300sec to get out of that state and start accepting the connections. 2. With the existing implementation without retries server dies down within 60sec. ``` INFO exited: backend (exit status 1; not expected) INFO gave up: backend entered FATAL state, too many start retries too quickly ``` Reference doc: https://www.notion.so/appsmith/Postgres-critical-scenarios-668f49c96aef40308e24c2a8d6b1137c /test Sanity ### :mag: Cypress test results <!-- This is an auto-generated comment: Cypress test results --> > [!TIP] > 🟢 🟢 🟢 All cypress tests have passed! 🎉 🎉 🎉 > Workflow run: <https://github.com/appsmithorg/appsmith/actions/runs/11100944184> > Commit: 9dbbe4b22ba12aa82c385ad0eef3cc3d4876f217 > <a href="https://internal.appsmith.com/app/cypress-dashboard/rundetails-65890b3c81d7400d08fa9ee5?branch=master&workflowId=11100944184&attempt=1" target="_blank">Cypress dashboard</a>. > Tags: `@tag.Sanity` > Spec: > <hr>Mon, 30 Sep 2024 07:26:41 UTC <!-- end of auto-generated comment: Cypress test results --> ## Communication Should the DevRel and Marketing teams inform users about this change? - [ ] Yes - [x] No <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit ## Summary by CodeRabbit - **New Features** - Introduced new functions for enhanced handling of PostgreSQL database connections, including availability checks and parameter extraction. - Added a new utility script for managing PostgreSQL connections. - **Bug Fixes** - Implemented a retry mechanism for PostgreSQL server availability checks to ensure more reliable connections. - **Tests** - Added unit tests to validate the functionality of the PostgreSQL parameter extraction logic. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
2024-09-30 07:27:47 +00:00
status=$?
case $status in
0)
tlog "PostgreSQL host '$PG_DB_HOST' is ready."
break
;;
1)
tlog "PostgreSQL host '$PG_DB_HOST' is rejecting connections e.g. due to being in recovery mode or not accepting connections eg. connections maxed out."
;;
2)
tlog "PostgreSQL host '$PG_DB_HOST' is not responding or running."
;;
3)
tlog "The connection check failed e.g. due to network issues or incorrect parameters."
;;
*)
tlog "pg_isready exited with unexpected status code: $status"
break
;;
esac
retry_count=$((retry_count + 1))
if [ $retry_count -le $MAX_RETRIES ]; then
tlog "PostgreSQL connection failed. Retrying attempt $retry_count/$MAX_RETRIES in $RETRYSECONDS seconds..."
sleep $RETRYSECONDS
else
tlog "Exceeded maximum retry attempts ($MAX_RETRIES). Exiting."
# use exit code 2 to indicate that the script failed to connect to postgres and supervisor conf is set not to restart the program for 2.
exit 2
fi
done
fi
}
# for PostgreSQL, we use APPSMITH_DB_URL=postgresql://username:password@postgresserver:5432/dbname
# Args:
# conn_string (string): PostgreSQL connection string
# Returns:
# None
# Example:
# postgres syntax
# "postgresql://user:password@localhost:5432/appsmith"
# "postgresql://user:password@localhost/appsmith"
# "postgresql://user@localhost:5432/appsmith"
# "postgresql://user@localhost/appsmith"
extract_postgres_db_params() {
local conn_string=$1
# Use node to parse the URI and extract components
IFS=' ' read -r USER PASSWORD HOST PORT DB <<<"$(node -e "
const connectionString = process.argv[1];
const pgUri = connectionString.startsWith(\"postgresql://\")
? connectionString
: 'http://' + connectionString; //Prepend a fake scheme for URL parsing
const url = require('url');
const parsedUrl = new url.URL(pgUri);
// Extract the pathname and remove the leading '/'
const db = parsedUrl.pathname.substring(1);
// Default the port to 5432 if it's empty
const port = parsedUrl.port || '5432';
console.log(\`\${parsedUrl.username || '-'} \${parsedUrl.password || '-'} \${parsedUrl.hostname} \${port} \${db}\`);
" "$conn_string")"
# Now, set the environment variables
export PG_DB_USER="$USER"
export PG_DB_PASSWORD="$PASSWORD"
export PG_DB_HOST="$HOST"
export PG_DB_PORT="$PORT"
export PG_DB_NAME="$DB"
}
chore: create appsmith schema for postgres (#36591) ## Description The current state is default schema or public schema. This schema is accessible by default when user connects to the pg database. Hence create `appsmith` schema for Appsmith server to use. This is to avoid anyone accidentally modifying the appsmith data. ## Automation /ok-to-test tags="@tag.Sanity" ### :mag: Cypress test results <!-- This is an auto-generated comment: Cypress test results --> > [!IMPORTANT] > 🟣 🟣 🟣 Your tests are running. > Tests running at: <https://github.com/appsmithorg/appsmith/actions/runs/11111681323> > Commit: 32f91e8d7ce750e4a088996aff4abe6905aa982f > Workflow: `PR Automation test suite` > Tags: `@tag.Sanity` > Spec: `` > <hr>Mon, 30 Sep 2024 18:08:23 UTC <!-- end of auto-generated comment: Cypress test results --> ## Communication Should the DevRel and Marketing teams inform users about this change? - [ ] Yes - [ ] No <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit ## Summary by CodeRabbit - **New Features** - Introduced a script to initialize the PostgreSQL database schema for Appsmith. - Added utilities for managing PostgreSQL database connections, including availability checks and parameter extraction. - Enhanced scripts for managing PostgreSQL connections and initialization. - Improved environment configuration for PostgreSQL database connections, including automatic password generation for local setups. - Updated JDBC URL handling to include schema parameters for PostgreSQL connections. - Added support for proxy configuration in the application setup. - **Bug Fixes** - Improved error handling and connection retry mechanisms for PostgreSQL setup. - **Documentation** - Updated comments and logging for better clarity on database operations. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Co-authored-by: Abhijeet <abhi.nagarnaik@gmail.com>
2024-09-30 18:12:56 +00:00
init_pg_db() {
# Create the appsmith schema
echo "Initializing PostgreSQL with schema..."
# Check if APPSMITH_DB_URL is a PostgreSQL URL
if [[ -n "$APPSMITH_DB_URL" && "$APPSMITH_DB_URL" == postgres*://* ]]; then
echo "APPSMITH_DB_URL is a valid PostgreSQL URL."
# Check if the DB_HOST is local (localhost or 127.0.0.1)
if [[ "$PG_DB_HOST" == "localhost" || "$PG_DB_HOST" == "127.0.0.1" ]]; then
local unix_socket_directory=$(get_unix_socket_directory "$POSTGRES_DB_PATH")
chore: create appsmith schema for postgres (#36591) ## Description The current state is default schema or public schema. This schema is accessible by default when user connects to the pg database. Hence create `appsmith` schema for Appsmith server to use. This is to avoid anyone accidentally modifying the appsmith data. ## Automation /ok-to-test tags="@tag.Sanity" ### :mag: Cypress test results <!-- This is an auto-generated comment: Cypress test results --> > [!IMPORTANT] > 🟣 🟣 🟣 Your tests are running. > Tests running at: <https://github.com/appsmithorg/appsmith/actions/runs/11111681323> > Commit: 32f91e8d7ce750e4a088996aff4abe6905aa982f > Workflow: `PR Automation test suite` > Tags: `@tag.Sanity` > Spec: `` > <hr>Mon, 30 Sep 2024 18:08:23 UTC <!-- end of auto-generated comment: Cypress test results --> ## Communication Should the DevRel and Marketing teams inform users about this change? - [ ] Yes - [ ] No <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit ## Summary by CodeRabbit - **New Features** - Introduced a script to initialize the PostgreSQL database schema for Appsmith. - Added utilities for managing PostgreSQL database connections, including availability checks and parameter extraction. - Enhanced scripts for managing PostgreSQL connections and initialization. - Improved environment configuration for PostgreSQL database connections, including automatic password generation for local setups. - Updated JDBC URL handling to include schema parameters for PostgreSQL connections. - Added support for proxy configuration in the application setup. - **Bug Fixes** - Improved error handling and connection retry mechanisms for PostgreSQL setup. - **Documentation** - Updated comments and logging for better clarity on database operations. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Co-authored-by: Abhijeet <abhi.nagarnaik@gmail.com>
2024-09-30 18:12:56 +00:00
# Check if the database exists
DB_CHECK=$(psql -h "$unix_socket_directory" -p "$PG_DB_PORT" -U "$POSTGRES_ADMIN_USER" -tAc "SELECT 1 FROM pg_database WHERE datname='$PG_DB_NAME'")
chore: create appsmith schema for postgres (#36591) ## Description The current state is default schema or public schema. This schema is accessible by default when user connects to the pg database. Hence create `appsmith` schema for Appsmith server to use. This is to avoid anyone accidentally modifying the appsmith data. ## Automation /ok-to-test tags="@tag.Sanity" ### :mag: Cypress test results <!-- This is an auto-generated comment: Cypress test results --> > [!IMPORTANT] > 🟣 🟣 🟣 Your tests are running. > Tests running at: <https://github.com/appsmithorg/appsmith/actions/runs/11111681323> > Commit: 32f91e8d7ce750e4a088996aff4abe6905aa982f > Workflow: `PR Automation test suite` > Tags: `@tag.Sanity` > Spec: `` > <hr>Mon, 30 Sep 2024 18:08:23 UTC <!-- end of auto-generated comment: Cypress test results --> ## Communication Should the DevRel and Marketing teams inform users about this change? - [ ] Yes - [ ] No <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit ## Summary by CodeRabbit - **New Features** - Introduced a script to initialize the PostgreSQL database schema for Appsmith. - Added utilities for managing PostgreSQL database connections, including availability checks and parameter extraction. - Enhanced scripts for managing PostgreSQL connections and initialization. - Improved environment configuration for PostgreSQL database connections, including automatic password generation for local setups. - Updated JDBC URL handling to include schema parameters for PostgreSQL connections. - Added support for proxy configuration in the application setup. - **Bug Fixes** - Improved error handling and connection retry mechanisms for PostgreSQL setup. - **Documentation** - Updated comments and logging for better clarity on database operations. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Co-authored-by: Abhijeet <abhi.nagarnaik@gmail.com>
2024-09-30 18:12:56 +00:00
if [ "$DB_CHECK" != "1" ]; then
echo "Database $PG_DB_NAME does not exist. Creating database..."
psql -h "$unix_socket_directory" -p "$PG_DB_PORT" -U "$POSTGRES_ADMIN_USER" -c "CREATE DATABASE $PG_DB_NAME;"
chore: create appsmith schema for postgres (#36591) ## Description The current state is default schema or public schema. This schema is accessible by default when user connects to the pg database. Hence create `appsmith` schema for Appsmith server to use. This is to avoid anyone accidentally modifying the appsmith data. ## Automation /ok-to-test tags="@tag.Sanity" ### :mag: Cypress test results <!-- This is an auto-generated comment: Cypress test results --> > [!IMPORTANT] > 🟣 🟣 🟣 Your tests are running. > Tests running at: <https://github.com/appsmithorg/appsmith/actions/runs/11111681323> > Commit: 32f91e8d7ce750e4a088996aff4abe6905aa982f > Workflow: `PR Automation test suite` > Tags: `@tag.Sanity` > Spec: `` > <hr>Mon, 30 Sep 2024 18:08:23 UTC <!-- end of auto-generated comment: Cypress test results --> ## Communication Should the DevRel and Marketing teams inform users about this change? - [ ] Yes - [ ] No <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit ## Summary by CodeRabbit - **New Features** - Introduced a script to initialize the PostgreSQL database schema for Appsmith. - Added utilities for managing PostgreSQL database connections, including availability checks and parameter extraction. - Enhanced scripts for managing PostgreSQL connections and initialization. - Improved environment configuration for PostgreSQL database connections, including automatic password generation for local setups. - Updated JDBC URL handling to include schema parameters for PostgreSQL connections. - Added support for proxy configuration in the application setup. - **Bug Fixes** - Improved error handling and connection retry mechanisms for PostgreSQL setup. - **Documentation** - Updated comments and logging for better clarity on database operations. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Co-authored-by: Abhijeet <abhi.nagarnaik@gmail.com>
2024-09-30 18:12:56 +00:00
else
echo "Database $PG_DB_NAME already exists."
fi
# Check if the schema exists
SCHEMA_CHECK=$(psql -h "$unix_socket_directory" -p "$PG_DB_PORT" -U "$POSTGRES_ADMIN_USER" -d "$PG_DB_NAME" -tAc "SELECT 1 FROM information_schema.schemata WHERE schema_name='appsmith'")
chore: create appsmith schema for postgres (#36591) ## Description The current state is default schema or public schema. This schema is accessible by default when user connects to the pg database. Hence create `appsmith` schema for Appsmith server to use. This is to avoid anyone accidentally modifying the appsmith data. ## Automation /ok-to-test tags="@tag.Sanity" ### :mag: Cypress test results <!-- This is an auto-generated comment: Cypress test results --> > [!IMPORTANT] > 🟣 🟣 🟣 Your tests are running. > Tests running at: <https://github.com/appsmithorg/appsmith/actions/runs/11111681323> > Commit: 32f91e8d7ce750e4a088996aff4abe6905aa982f > Workflow: `PR Automation test suite` > Tags: `@tag.Sanity` > Spec: `` > <hr>Mon, 30 Sep 2024 18:08:23 UTC <!-- end of auto-generated comment: Cypress test results --> ## Communication Should the DevRel and Marketing teams inform users about this change? - [ ] Yes - [ ] No <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit ## Summary by CodeRabbit - **New Features** - Introduced a script to initialize the PostgreSQL database schema for Appsmith. - Added utilities for managing PostgreSQL database connections, including availability checks and parameter extraction. - Enhanced scripts for managing PostgreSQL connections and initialization. - Improved environment configuration for PostgreSQL database connections, including automatic password generation for local setups. - Updated JDBC URL handling to include schema parameters for PostgreSQL connections. - Added support for proxy configuration in the application setup. - **Bug Fixes** - Improved error handling and connection retry mechanisms for PostgreSQL setup. - **Documentation** - Updated comments and logging for better clarity on database operations. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Co-authored-by: Abhijeet <abhi.nagarnaik@gmail.com>
2024-09-30 18:12:56 +00:00
# Create schema and user if not exists
if [ "$SCHEMA_CHECK" != "1" ]; then
echo "Creating user '$PG_DB_USER' with password "
psql -h "$unix_socket_directory" -p "$PG_DB_PORT" -U "$POSTGRES_ADMIN_USER" -d "$PG_DB_NAME" -c "CREATE USER \"$PG_DB_USER\" WITH PASSWORD '$PG_DB_PASSWORD';"
chore: create appsmith schema for postgres (#36591) ## Description The current state is default schema or public schema. This schema is accessible by default when user connects to the pg database. Hence create `appsmith` schema for Appsmith server to use. This is to avoid anyone accidentally modifying the appsmith data. ## Automation /ok-to-test tags="@tag.Sanity" ### :mag: Cypress test results <!-- This is an auto-generated comment: Cypress test results --> > [!IMPORTANT] > 🟣 🟣 🟣 Your tests are running. > Tests running at: <https://github.com/appsmithorg/appsmith/actions/runs/11111681323> > Commit: 32f91e8d7ce750e4a088996aff4abe6905aa982f > Workflow: `PR Automation test suite` > Tags: `@tag.Sanity` > Spec: `` > <hr>Mon, 30 Sep 2024 18:08:23 UTC <!-- end of auto-generated comment: Cypress test results --> ## Communication Should the DevRel and Marketing teams inform users about this change? - [ ] Yes - [ ] No <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit ## Summary by CodeRabbit - **New Features** - Introduced a script to initialize the PostgreSQL database schema for Appsmith. - Added utilities for managing PostgreSQL database connections, including availability checks and parameter extraction. - Enhanced scripts for managing PostgreSQL connections and initialization. - Improved environment configuration for PostgreSQL database connections, including automatic password generation for local setups. - Updated JDBC URL handling to include schema parameters for PostgreSQL connections. - Added support for proxy configuration in the application setup. - **Bug Fixes** - Improved error handling and connection retry mechanisms for PostgreSQL setup. - **Documentation** - Updated comments and logging for better clarity on database operations. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Co-authored-by: Abhijeet <abhi.nagarnaik@gmail.com>
2024-09-30 18:12:56 +00:00
echo "Schema 'appsmith' does not exist. Creating schema..."
psql -h "$unix_socket_directory" -p "$PG_DB_PORT" -U "$POSTGRES_ADMIN_USER" -d "$PG_DB_NAME" -c "CREATE SCHEMA appsmith;"
chore: create appsmith schema for postgres (#36591) ## Description The current state is default schema or public schema. This schema is accessible by default when user connects to the pg database. Hence create `appsmith` schema for Appsmith server to use. This is to avoid anyone accidentally modifying the appsmith data. ## Automation /ok-to-test tags="@tag.Sanity" ### :mag: Cypress test results <!-- This is an auto-generated comment: Cypress test results --> > [!IMPORTANT] > 🟣 🟣 🟣 Your tests are running. > Tests running at: <https://github.com/appsmithorg/appsmith/actions/runs/11111681323> > Commit: 32f91e8d7ce750e4a088996aff4abe6905aa982f > Workflow: `PR Automation test suite` > Tags: `@tag.Sanity` > Spec: `` > <hr>Mon, 30 Sep 2024 18:08:23 UTC <!-- end of auto-generated comment: Cypress test results --> ## Communication Should the DevRel and Marketing teams inform users about this change? - [ ] Yes - [ ] No <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit ## Summary by CodeRabbit - **New Features** - Introduced a script to initialize the PostgreSQL database schema for Appsmith. - Added utilities for managing PostgreSQL database connections, including availability checks and parameter extraction. - Enhanced scripts for managing PostgreSQL connections and initialization. - Improved environment configuration for PostgreSQL database connections, including automatic password generation for local setups. - Updated JDBC URL handling to include schema parameters for PostgreSQL connections. - Added support for proxy configuration in the application setup. - **Bug Fixes** - Improved error handling and connection retry mechanisms for PostgreSQL setup. - **Documentation** - Updated comments and logging for better clarity on database operations. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Co-authored-by: Abhijeet <abhi.nagarnaik@gmail.com>
2024-09-30 18:12:56 +00:00
fi
echo "Creating pg_trgm extension..."
psql -h "$unix_socket_directory" -p "$PG_DB_PORT" -U "$POSTGRES_ADMIN_USER" -d "$PG_DB_NAME" -c "CREATE EXTENSION IF NOT EXISTS pg_trgm;"
# Grant permissions to the user on the schema
USER=$PG_DB_USER SCHEMA="appsmith" DB=$PG_DB_NAME HOST=$PG_DB_HOST PORT=$PG_DB_PORT grant_permissions_for_local_db_schema
chore: create appsmith schema for postgres (#36591) ## Description The current state is default schema or public schema. This schema is accessible by default when user connects to the pg database. Hence create `appsmith` schema for Appsmith server to use. This is to avoid anyone accidentally modifying the appsmith data. ## Automation /ok-to-test tags="@tag.Sanity" ### :mag: Cypress test results <!-- This is an auto-generated comment: Cypress test results --> > [!IMPORTANT] > 🟣 🟣 🟣 Your tests are running. > Tests running at: <https://github.com/appsmithorg/appsmith/actions/runs/11111681323> > Commit: 32f91e8d7ce750e4a088996aff4abe6905aa982f > Workflow: `PR Automation test suite` > Tags: `@tag.Sanity` > Spec: `` > <hr>Mon, 30 Sep 2024 18:08:23 UTC <!-- end of auto-generated comment: Cypress test results --> ## Communication Should the DevRel and Marketing teams inform users about this change? - [ ] Yes - [ ] No <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit ## Summary by CodeRabbit - **New Features** - Introduced a script to initialize the PostgreSQL database schema for Appsmith. - Added utilities for managing PostgreSQL database connections, including availability checks and parameter extraction. - Enhanced scripts for managing PostgreSQL connections and initialization. - Improved environment configuration for PostgreSQL database connections, including automatic password generation for local setups. - Updated JDBC URL handling to include schema parameters for PostgreSQL connections. - Added support for proxy configuration in the application setup. - **Bug Fixes** - Improved error handling and connection retry mechanisms for PostgreSQL setup. - **Documentation** - Updated comments and logging for better clarity on database operations. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Co-authored-by: Abhijeet <abhi.nagarnaik@gmail.com>
2024-09-30 18:12:56 +00:00
else
echo "Remote PostgreSQL detected, running as current user."
PGPASSWORD=$PG_DB_PASSWORD psql -h "$PG_DB_HOST" -p "$PG_DB_PORT" -U "$PG_DB_USER" -d "$PG_DB_NAME" -c "CREATE SCHEMA IF NOT EXISTS appsmith;"
echo "Creating pg_trgm extension..."
PGPASSWORD=$PG_DB_PASSWORD psql -h "$PG_DB_HOST" -p "$PG_DB_PORT" -U "$PG_DB_USER" -d "$PG_DB_NAME" -c "CREATE EXTENSION IF NOT EXISTS pg_trgm;"
chore: create appsmith schema for postgres (#36591) ## Description The current state is default schema or public schema. This schema is accessible by default when user connects to the pg database. Hence create `appsmith` schema for Appsmith server to use. This is to avoid anyone accidentally modifying the appsmith data. ## Automation /ok-to-test tags="@tag.Sanity" ### :mag: Cypress test results <!-- This is an auto-generated comment: Cypress test results --> > [!IMPORTANT] > 🟣 🟣 🟣 Your tests are running. > Tests running at: <https://github.com/appsmithorg/appsmith/actions/runs/11111681323> > Commit: 32f91e8d7ce750e4a088996aff4abe6905aa982f > Workflow: `PR Automation test suite` > Tags: `@tag.Sanity` > Spec: `` > <hr>Mon, 30 Sep 2024 18:08:23 UTC <!-- end of auto-generated comment: Cypress test results --> ## Communication Should the DevRel and Marketing teams inform users about this change? - [ ] Yes - [ ] No <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit ## Summary by CodeRabbit - **New Features** - Introduced a script to initialize the PostgreSQL database schema for Appsmith. - Added utilities for managing PostgreSQL database connections, including availability checks and parameter extraction. - Enhanced scripts for managing PostgreSQL connections and initialization. - Improved environment configuration for PostgreSQL database connections, including automatic password generation for local setups. - Updated JDBC URL handling to include schema parameters for PostgreSQL connections. - Added support for proxy configuration in the application setup. - **Bug Fixes** - Improved error handling and connection retry mechanisms for PostgreSQL setup. - **Documentation** - Updated comments and logging for better clarity on database operations. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Co-authored-by: Abhijeet <abhi.nagarnaik@gmail.com>
2024-09-30 18:12:56 +00:00
fi
# Check if the schema creation was successful
if [ $? -eq 0 ]; then
echo "Schema 'appsmith' created or already exists."
else
echo "Failed to create schema 'appsmith'."
exit 1
fi
echo "PostgreSQL initialization completed."
fi
}
chore: Add grants for appsmith user for embedded postgres (#36664) ## Description PR to add the necessary grants to `appsmith` user when user opts for Postgres embedded DB. fixes https://github.com/appsmithorg/appsmith/issues/36661 ## Automation /test Sanity ### :mag: Cypress test results <!-- This is an auto-generated comment: Cypress test results --> > [!TIP] > 🟢 🟢 🟢 All cypress tests have passed! 🎉 🎉 🎉 > Workflow run: <https://github.com/appsmithorg/appsmith/actions/runs/11155064003> > Commit: 1fb82e52a3f05da1b9bb4082cf871a62d3554d78 > <a href="https://internal.appsmith.com/app/cypress-dashboard/rundetails-65890b3c81d7400d08fa9ee5?branch=master&workflowId=11155064003&attempt=2" target="_blank">Cypress dashboard</a>. > Tags: `@tag.Sanity` > Spec: > <hr>Thu, 03 Oct 2024 04:22:54 UTC <!-- end of auto-generated comment: Cypress test results --> ## Communication Should the DevRel and Marketing teams inform users about this change? - [ ] Yes - [x] No <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit - **New Features** - Introduced default values for PostgreSQL database connection parameters. - Added a new function to manage user permissions on database schemas. - **Improvements** - Enhanced the existing database initialization process to include permission granting after schema verification. - Updated documentation within the script for better clarity on new functionalities. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
2024-10-04 04:47:17 +00:00
# Utility function to grant permissions to a user on a schema in a database on a host and port in PostgreSQL database
# Args:
# USER (string): User to grant permissions to
# SCHEMA (string): Schema to grant permissions on
# DB (string): Database to grant permissions on
# HOST (string): Host to grant permissions on
# PORT (int): Port to grant permissions on
# Returns:
# None
# Example:
# USER="user" SCHEMA="schema" DB="db" HOST="host" PORT="port" grant_permissions_for_local_db_schema
grant_permissions_for_local_db_schema() {
chore: Add grants for appsmith user for embedded postgres (#36664) ## Description PR to add the necessary grants to `appsmith` user when user opts for Postgres embedded DB. fixes https://github.com/appsmithorg/appsmith/issues/36661 ## Automation /test Sanity ### :mag: Cypress test results <!-- This is an auto-generated comment: Cypress test results --> > [!TIP] > 🟢 🟢 🟢 All cypress tests have passed! 🎉 🎉 🎉 > Workflow run: <https://github.com/appsmithorg/appsmith/actions/runs/11155064003> > Commit: 1fb82e52a3f05da1b9bb4082cf871a62d3554d78 > <a href="https://internal.appsmith.com/app/cypress-dashboard/rundetails-65890b3c81d7400d08fa9ee5?branch=master&workflowId=11155064003&attempt=2" target="_blank">Cypress dashboard</a>. > Tags: `@tag.Sanity` > Spec: > <hr>Thu, 03 Oct 2024 04:22:54 UTC <!-- end of auto-generated comment: Cypress test results --> ## Communication Should the DevRel and Marketing teams inform users about this change? - [ ] Yes - [x] No <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit - **New Features** - Introduced default values for PostgreSQL database connection parameters. - Added a new function to manage user permissions on database schemas. - **Improvements** - Enhanced the existing database initialization process to include permission granting after schema verification. - Updated documentation within the script for better clarity on new functionalities. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
2024-10-04 04:47:17 +00:00
local user=${USER-$DB_USER} schema=${SCHEMA-$DB_SCHEMA} db=${DB-$DB_NAME} host=${HOST-$DB_HOST} port=${PORT-$DB_PORT}
local unix_socket_directory=$(get_unix_socket_directory "$POSTGRES_DB_PATH")
chore: Add grants for appsmith user for embedded postgres (#36664) ## Description PR to add the necessary grants to `appsmith` user when user opts for Postgres embedded DB. fixes https://github.com/appsmithorg/appsmith/issues/36661 ## Automation /test Sanity ### :mag: Cypress test results <!-- This is an auto-generated comment: Cypress test results --> > [!TIP] > 🟢 🟢 🟢 All cypress tests have passed! 🎉 🎉 🎉 > Workflow run: <https://github.com/appsmithorg/appsmith/actions/runs/11155064003> > Commit: 1fb82e52a3f05da1b9bb4082cf871a62d3554d78 > <a href="https://internal.appsmith.com/app/cypress-dashboard/rundetails-65890b3c81d7400d08fa9ee5?branch=master&workflowId=11155064003&attempt=2" target="_blank">Cypress dashboard</a>. > Tags: `@tag.Sanity` > Spec: > <hr>Thu, 03 Oct 2024 04:22:54 UTC <!-- end of auto-generated comment: Cypress test results --> ## Communication Should the DevRel and Marketing teams inform users about this change? - [ ] Yes - [x] No <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit - **New Features** - Introduced default values for PostgreSQL database connection parameters. - Added a new function to manage user permissions on database schemas. - **Improvements** - Enhanced the existing database initialization process to include permission granting after schema verification. - Updated documentation within the script for better clarity on new functionalities. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
2024-10-04 04:47:17 +00:00
tlog "Granting permissions to user '${user}' on schema '$schema' in database '$db' on host '$host' and port '$port'..."
psql -h "$unix_socket_directory" -p "$port" -U "$POSTGRES_ADMIN_USER" -d "$db" -c "GRANT ALL PRIVILEGES ON SCHEMA ${schema} TO ${user};"
psql -h "$unix_socket_directory" -p "$port" -U "$POSTGRES_ADMIN_USER" -d "$db" -c "GRANT ALL PRIVILEGES ON ALL TABLES IN SCHEMA ${schema} TO ${user};"
psql -h "$unix_socket_directory" -p "$port" -U "$POSTGRES_ADMIN_USER" -d "$db" -c "ALTER DEFAULT PRIVILEGES IN SCHEMA ${schema} GRANT ALL PRIVILEGES ON TABLES TO ${user};"
psql -h "$unix_socket_directory" -p "$port" -U "$POSTGRES_ADMIN_USER" -d "$db" -c "GRANT CONNECT ON DATABASE ${db} TO ${user};"
}
get_unix_socket_directory() {
local postgres_db_path=${1:-"$POSTGRES_DB_PATH"}
local unix_socket_directory
unix_socket_directory=$(grep -E "^unix_socket_directories" "$postgres_db_path/postgresql.conf" | sed -E "s/.*= (.*).*/\1/" | cut -d',' -f1)
# If unix_socket_directory is empty, default to /var/run/postgresql
if [ -z "$unix_socket_directory" ]; then
unix_socket_directory="/var/run/postgresql"
fi
echo "$unix_socket_directory"
chore: Add grants for appsmith user for embedded postgres (#36664) ## Description PR to add the necessary grants to `appsmith` user when user opts for Postgres embedded DB. fixes https://github.com/appsmithorg/appsmith/issues/36661 ## Automation /test Sanity ### :mag: Cypress test results <!-- This is an auto-generated comment: Cypress test results --> > [!TIP] > 🟢 🟢 🟢 All cypress tests have passed! 🎉 🎉 🎉 > Workflow run: <https://github.com/appsmithorg/appsmith/actions/runs/11155064003> > Commit: 1fb82e52a3f05da1b9bb4082cf871a62d3554d78 > <a href="https://internal.appsmith.com/app/cypress-dashboard/rundetails-65890b3c81d7400d08fa9ee5?branch=master&workflowId=11155064003&attempt=2" target="_blank">Cypress dashboard</a>. > Tags: `@tag.Sanity` > Spec: > <hr>Thu, 03 Oct 2024 04:22:54 UTC <!-- end of auto-generated comment: Cypress test results --> ## Communication Should the DevRel and Marketing teams inform users about this change? - [ ] Yes - [x] No <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit - **New Features** - Introduced default values for PostgreSQL database connection parameters. - Added a new function to manage user permissions on database schemas. - **Improvements** - Enhanced the existing database initialization process to include permission granting after schema verification. - Updated documentation within the script for better clarity on new functionalities. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
2024-10-04 04:47:17 +00:00
}
chore: Add postgres dependency for server to startup (#36585) ## Description As in the past we have seen the corruption of postgres DB which is being used for temporal we want to make sure we have a retry mechanism in place if: 1. `APPSMITH_DB_URL` is pointing to postgres url 2. Postgres is waiting in recovery mode As per local testing when the docker container is abruptly shutdown via `docker rm -f {container_name}` or `docker kill {container_name}` or even via docker desktop we end up in state where postgres goes into recovery state. logs: ``` 2024-09-27 08:02:49 backend stdout | SQL State : 57P03 2024-09-27 08:02:49 backend stdout | Error Code : 0 2024-09-27 08:02:49 backend stdout | Message : FATAL: the database system is starting up ``` Currently we have implemented polling mechanism, but we will keep looking for better alternative here if we can opt for. Note: 1. With release dump this is taking ~300sec to get out of that state and start accepting the connections. 2. With the existing implementation without retries server dies down within 60sec. ``` INFO exited: backend (exit status 1; not expected) INFO gave up: backend entered FATAL state, too many start retries too quickly ``` Reference doc: https://www.notion.so/appsmith/Postgres-critical-scenarios-668f49c96aef40308e24c2a8d6b1137c /test Sanity ### :mag: Cypress test results <!-- This is an auto-generated comment: Cypress test results --> > [!TIP] > 🟢 🟢 🟢 All cypress tests have passed! 🎉 🎉 🎉 > Workflow run: <https://github.com/appsmithorg/appsmith/actions/runs/11100944184> > Commit: 9dbbe4b22ba12aa82c385ad0eef3cc3d4876f217 > <a href="https://internal.appsmith.com/app/cypress-dashboard/rundetails-65890b3c81d7400d08fa9ee5?branch=master&workflowId=11100944184&attempt=1" target="_blank">Cypress dashboard</a>. > Tags: `@tag.Sanity` > Spec: > <hr>Mon, 30 Sep 2024 07:26:41 UTC <!-- end of auto-generated comment: Cypress test results --> ## Communication Should the DevRel and Marketing teams inform users about this change? - [ ] Yes - [x] No <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit ## Summary by CodeRabbit - **New Features** - Introduced new functions for enhanced handling of PostgreSQL database connections, including availability checks and parameter extraction. - Added a new utility script for managing PostgreSQL connections. - **Bug Fixes** - Implemented a retry mechanism for PostgreSQL server availability checks to ensure more reliable connections. - **Tests** - Added unit tests to validate the functionality of the PostgreSQL parameter extraction logic. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
2024-09-30 07:27:47 +00:00
# Example usage of the functions
# waitForPostgresAvailability
chore: create appsmith schema for postgres (#36591) ## Description The current state is default schema or public schema. This schema is accessible by default when user connects to the pg database. Hence create `appsmith` schema for Appsmith server to use. This is to avoid anyone accidentally modifying the appsmith data. ## Automation /ok-to-test tags="@tag.Sanity" ### :mag: Cypress test results <!-- This is an auto-generated comment: Cypress test results --> > [!IMPORTANT] > 🟣 🟣 🟣 Your tests are running. > Tests running at: <https://github.com/appsmithorg/appsmith/actions/runs/11111681323> > Commit: 32f91e8d7ce750e4a088996aff4abe6905aa982f > Workflow: `PR Automation test suite` > Tags: `@tag.Sanity` > Spec: `` > <hr>Mon, 30 Sep 2024 18:08:23 UTC <!-- end of auto-generated comment: Cypress test results --> ## Communication Should the DevRel and Marketing teams inform users about this change? - [ ] Yes - [ ] No <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit ## Summary by CodeRabbit - **New Features** - Introduced a script to initialize the PostgreSQL database schema for Appsmith. - Added utilities for managing PostgreSQL database connections, including availability checks and parameter extraction. - Enhanced scripts for managing PostgreSQL connections and initialization. - Improved environment configuration for PostgreSQL database connections, including automatic password generation for local setups. - Updated JDBC URL handling to include schema parameters for PostgreSQL connections. - Added support for proxy configuration in the application setup. - **Bug Fixes** - Improved error handling and connection retry mechanisms for PostgreSQL setup. - **Documentation** - Updated comments and logging for better clarity on database operations. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Co-authored-by: Abhijeet <abhi.nagarnaik@gmail.com>
2024-09-30 18:12:56 +00:00
# extract_postgres_db_params "postgresql://user:password@localhost:5432/dbname"
chore: Add grants for appsmith user for embedded postgres (#36664) ## Description PR to add the necessary grants to `appsmith` user when user opts for Postgres embedded DB. fixes https://github.com/appsmithorg/appsmith/issues/36661 ## Automation /test Sanity ### :mag: Cypress test results <!-- This is an auto-generated comment: Cypress test results --> > [!TIP] > 🟢 🟢 🟢 All cypress tests have passed! 🎉 🎉 🎉 > Workflow run: <https://github.com/appsmithorg/appsmith/actions/runs/11155064003> > Commit: 1fb82e52a3f05da1b9bb4082cf871a62d3554d78 > <a href="https://internal.appsmith.com/app/cypress-dashboard/rundetails-65890b3c81d7400d08fa9ee5?branch=master&workflowId=11155064003&attempt=2" target="_blank">Cypress dashboard</a>. > Tags: `@tag.Sanity` > Spec: > <hr>Thu, 03 Oct 2024 04:22:54 UTC <!-- end of auto-generated comment: Cypress test results --> ## Communication Should the DevRel and Marketing teams inform users about this change? - [ ] Yes - [x] No <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit - **New Features** - Introduced default values for PostgreSQL database connection parameters. - Added a new function to manage user permissions on database schemas. - **Improvements** - Enhanced the existing database initialization process to include permission granting after schema verification. - Updated documentation within the script for better clarity on new functionalities. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
2024-10-04 04:47:17 +00:00
# init_pg_db
# USER="user" SCHEMA="schema" DB="db" HOST="host" PORT="port" grant_permissions_for_local_db_schema
# get_unix_socket_directory "/var/lib/postgresql/12/main"