## Description
We introduced the auth for embedded Postgres, but in pg-utils we missed
on updating the pg_isready condition to resort to db host in case of
external postgres url. This caused the pg_isready to never pass when
user configure external postgres url as a persistence DB in Appsmith.
This PR checks if the local postgres url is configured and accordingly
modifies the pg_isready command to target the embedded host vs external
host.
Why it did not get caught in our testing?
We are still investigating this but in local for some reason even if we
opt for external Postgres DB embedded Postgres process is also
initiating which runs the pg_isready against the embedded Postgres and
startup goes through.
Fixes #`Issue Number`
_or_
Fixes `Issue URL`
> [!WARNING]
> _If no issue exists, please create an issue first, and check with the
maintainers if the issue is valid._
## Automation
/ok-to-test tags=""
### 🔍 Cypress test results
<!-- This is an auto-generated comment: Cypress test results -->
> [!CAUTION]
> If you modify the content in this section, you are likely to disrupt
the CI result for your PR.
<!-- end of auto-generated comment: Cypress test results -->
## Communication
Should the DevRel and Marketing teams inform users about this change?
- [ ] Yes
- [ ] No
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Enhanced PostgreSQL connection handling with dynamic checks for local
or remote instances.
- Improved flexibility in connection strategy for better database
accessibility.
- **Bug Fixes**
- Updated connection commands to ensure proper usage of the new host
proxy variable.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## Description
Use password while creating pg_trm extension on remote postgres.
Fixes #`Issue Number`
_or_
Fixes `Issue URL`
> [!WARNING]
> _If no issue exists, please create an issue first, and check with the
maintainers if the issue is valid._
## Automation
/ok-to-test tags=""
### 🔍 Cypress test results
<!-- This is an auto-generated comment: Cypress test results -->
> [!CAUTION]
> If you modify the content in this section, you are likely to disrupt
the CI result for your PR.
<!-- end of auto-generated comment: Cypress test results -->
## Communication
Should the DevRel and Marketing teams inform users about this change?
- [ ] Yes
- [ ] No
## Description
Due to the size limit on the BTree index we are creating GIN index for a
permission group. To support this we need to create a pg_trm extension.
## Automation
/ok-to-test tags="@tag.Sanity"
### 🔍 Cypress test results
<!-- This is an auto-generated comment: Cypress test results -->
> [!TIP]
> 🟢🟢🟢 All cypress tests have passed! 🎉🎉🎉
> Workflow run:
<https://github.com/appsmithorg/appsmith/actions/runs/11212861701>
> Commit: d62be8f46ad45b1190bf5c014c0a8ea0c50cf284
> <a
href="https://internal.appsmith.com/app/cypress-dashboard/rundetails-65890b3c81d7400d08fa9ee5?branch=master&workflowId=11212861701&attempt=2"
target="_blank">Cypress dashboard</a>.
> Tags: `@tag.Sanity`
> Spec:
> <hr>Mon, 07 Oct 2024 10:46:39 UTC
<!-- end of auto-generated comment: Cypress test results -->
## Communication
Should the DevRel and Marketing teams inform users about this change?
- [ ] Yes
- [ ] No
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Enhanced database initialization by adding the creation of the
`pg_trgm` extension for PostgreSQL, improving text search capabilities.
- **Bug Fixes**
- Ensured the extension is created only if it does not already exist,
preventing unnecessary errors during setup.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## Description
PR to add the necessary grants to `appsmith` user when user opts for Postgres embedded DB.
fixes https://github.com/appsmithorg/appsmith/issues/36661
## Automation
/test Sanity
### 🔍 Cypress test results
<!-- This is an auto-generated comment: Cypress test results -->
> [!TIP]
> 🟢🟢🟢 All cypress tests have passed! 🎉🎉🎉
> Workflow run:
<https://github.com/appsmithorg/appsmith/actions/runs/11155064003>
> Commit: 1fb82e52a3f05da1b9bb4082cf871a62d3554d78
> <a
href="https://internal.appsmith.com/app/cypress-dashboard/rundetails-65890b3c81d7400d08fa9ee5?branch=master&workflowId=11155064003&attempt=2"
target="_blank">Cypress dashboard</a>.
> Tags: `@tag.Sanity`
> Spec:
> <hr>Thu, 03 Oct 2024 04:22:54 UTC
<!-- end of auto-generated comment: Cypress test results -->
## Communication
Should the DevRel and Marketing teams inform users about this change?
- [ ] Yes
- [x] No
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Introduced default values for PostgreSQL database connection
parameters.
- Added a new function to manage user permissions on database schemas.
- **Improvements**
- Enhanced the existing database initialization process to include
permission granting after schema verification.
- Updated documentation within the script for better clarity on new
functionalities.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## Description
The current state is default schema or public schema. This schema is
accessible by default when user connects to the pg database. Hence
create `appsmith` schema for Appsmith server to use. This is to avoid
anyone accidentally modifying the appsmith data.
## Automation
/ok-to-test tags="@tag.Sanity"
### 🔍 Cypress test results
<!-- This is an auto-generated comment: Cypress test results -->
> [!IMPORTANT]
> 🟣🟣🟣 Your tests are running.
> Tests running at:
<https://github.com/appsmithorg/appsmith/actions/runs/11111681323>
> Commit: 32f91e8d7ce750e4a088996aff4abe6905aa982f
> Workflow: `PR Automation test suite`
> Tags: `@tag.Sanity`
> Spec: ``
> <hr>Mon, 30 Sep 2024 18:08:23 UTC
<!-- end of auto-generated comment: Cypress test results -->
## Communication
Should the DevRel and Marketing teams inform users about this change?
- [ ] Yes
- [ ] No
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
## Summary by CodeRabbit
- **New Features**
- Introduced a script to initialize the PostgreSQL database schema for
Appsmith.
- Added utilities for managing PostgreSQL database connections,
including availability checks and parameter extraction.
- Enhanced scripts for managing PostgreSQL connections and
initialization.
- Improved environment configuration for PostgreSQL database
connections, including automatic password generation for local setups.
- Updated JDBC URL handling to include schema parameters for PostgreSQL
connections.
- Added support for proxy configuration in the application setup.
- **Bug Fixes**
- Improved error handling and connection retry mechanisms for PostgreSQL
setup.
- **Documentation**
- Updated comments and logging for better clarity on database
operations.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: Abhijeet <abhi.nagarnaik@gmail.com>
## Description
As in the past we have seen the corruption of postgres DB which is being
used for temporal we want to make sure we have a retry mechanism in
place if:
1. `APPSMITH_DB_URL` is pointing to postgres url
2. Postgres is waiting in recovery mode
As per local testing when the docker container is abruptly shutdown via
`docker rm -f {container_name}` or `docker kill {container_name}` or
even via docker desktop we end up in state where postgres goes into
recovery state.
logs:
```
2024-09-27 08:02:49 backend stdout | SQL State : 57P03
2024-09-27 08:02:49 backend stdout | Error Code : 0
2024-09-27 08:02:49 backend stdout | Message : FATAL: the database system is starting up
```
Currently we have implemented polling mechanism, but we will keep
looking for better alternative here if we can opt for.
Note:
1. With release dump this is taking ~300sec to get out of that state and
start accepting the connections.
2. With the existing implementation without retries server dies down
within 60sec.
```
INFO exited: backend (exit status 1; not expected)
INFO gave up: backend entered FATAL state, too many start retries too quickly
```
Reference doc:
https://www.notion.so/appsmith/Postgres-critical-scenarios-668f49c96aef40308e24c2a8d6b1137c
/test Sanity
### 🔍 Cypress test results
<!-- This is an auto-generated comment: Cypress test results -->
> [!TIP]
> 🟢🟢🟢 All cypress tests have passed! 🎉🎉🎉
> Workflow run:
<https://github.com/appsmithorg/appsmith/actions/runs/11100944184>
> Commit: 9dbbe4b22ba12aa82c385ad0eef3cc3d4876f217
> <a
href="https://internal.appsmith.com/app/cypress-dashboard/rundetails-65890b3c81d7400d08fa9ee5?branch=master&workflowId=11100944184&attempt=1"
target="_blank">Cypress dashboard</a>.
> Tags: `@tag.Sanity`
> Spec:
> <hr>Mon, 30 Sep 2024 07:26:41 UTC
<!-- end of auto-generated comment: Cypress test results -->
## Communication
Should the DevRel and Marketing teams inform users about this change?
- [ ] Yes
- [x] No
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
## Summary by CodeRabbit
- **New Features**
- Introduced new functions for enhanced handling of PostgreSQL database
connections, including availability checks and parameter extraction.
- Added a new utility script for managing PostgreSQL connections.
- **Bug Fixes**
- Implemented a retry mechanism for PostgreSQL server availability
checks to ensure more reliable connections.
- **Tests**
- Added unit tests to validate the functionality of the PostgreSQL
parameter extraction logic.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
