Now we have a lot of flexibility around the SecurityContext during testing. We can instantiate any type of mock users bearing different roles & permissions.
* Adding generic get(Multivalue<String, String> params) implementation to the BaseService. Now all the domain obejcts can simply leverage this base implementation for get queries out of the box for all the fields.
* IMP: For some reason, the query by example has stopped working across the board. Even the super implementation in SimpleReactiveMongoRepository doesn't work. No idea why this is happening. For the time being, have moved to Criteria queries for the get request.
Mongock is a fork of Mongobee with a better integration with
Spring. It also does not suffer from trying to access the
`system.indexes` collection.
See https://github.com/mongobee/mongobee/pull/87.
This ensures that the client can change the config at will without depending on the platform team to add the field to the POJO.
Also deleting redundant classes and controllers for propertyPane and Widget. These are no longer required and can be removed from the codebase.
Also fixing the SeedMongoData file with the seed data to correspond to the changes made in the OrganizationRepository. We will default to using the mongoTemplate in the SeedMongoData file in the future so that we don't have to deal with ACL when we are simply populating the DB.
Also adding test case for get applications. Now we will fetch all applications that the user has read permissions to by default. It's not dependent on the organization that they are a part of.
2. Add to Page changes : Documentation object has been added in Action to handle the extra Template documentation for actions that have been imported from 3p marketplace
3. Added basic structure for rapid api plugin by copy pasting the rest api plugin
These graphs help us map policies that are inherited from the parent and also lateral policies that are assigned to the users given that the user has a particular permission. Currently, the hierarchy has been defined for org & application. Need to cascade it to more documents such as pages & actions.
Correcting the writeKey for Segment in Staging
Also adding Mono.cache in the user invite flow to ensure that the same Mono is not executed multiple times.
See merge request theappsmith/internal-tools-server!224
2. Add to Page changes : Documentation object has been added in Action to handle the extra Template documentation for actions that have been imported from 3p marketplace
3. Added basic structure for rapid api plugin by copy pasting the rest api plugin
2. Add to Page changes : Documentation object has been added in Action to handle the extra Template documentation for actions that have been imported from 3p marketplace
3. Added basic structure for rapid api plugin by copy pasting the rest api plugin
Also adding the Origin header to the BaseController create function. This is required by the user creation flow in order to customize the links in the email. For most of the controllers overriding the BaseController, the request header parameter is non-mandatory and can be skipped for testing or otherwise.
In order to create an application with valid permissions, we copy the relevant permissions from the organization of the user. This involves changes to the organizationService and OrganizationRepository as well.
Also moving the argument AclPermission to the custom Repository interface level. This is to ensure that all the service functions can invoke the same repository function with different permissions based on their requirements.
This is present in the BaseRepositoryImpl. In this manner, we can add default conditions to all the basic JPA queries. Now we only need to figure out how to intercept the custom queries to add user details into.
This is to ensure that setting the createdAt nad updatedAt properties by spring-data will still succeed even if the client doesn't provide us with the datasource details.
Creating an embedded datasource for the action
Now the user can create an embedded datasource as well as a saved datasource for the action.
This MR also fixes the import for a curl command by setting defaults when an action is created via the curl command. For now, the curl command only accepts short flags and not the long form flags.
See merge request theappsmith/internal-tools-server!204
We now create another list from the value provided in the properties file. All checks in the codebase are performed against this list. This ensures that there are no NPE and exceptions when the property oauth2.allowed-domains is removed from the properties file.
Correcting the error response returned by the AclFilter to match the ErrorDTO...
Correcting the error response returned by the AclFilter to match the ErrorDTO returned from controller functions
This ensures consistent responses from different parts of our application. The client can then rely on the server to provide a constistent response structure.
See merge request theappsmith/internal-tools-server!194
This ensures consistent responses from different parts of our application. The client can then rely on the server to provide a constistent response structure.
Renaming isPaginated to PaginationType to support different types of pagination on the frontend
If the API is paginated via the table's page number, then the next and prev URLs in the action don't play any role. Those fields are only required when the paginationType is of type URL. In all other scenarios, the action is executed normally.
See merge request theappsmith/internal-tools-server!192
If the API is paginated via the table's page number, then the next and prev URLs in the action don't play any role. Those fields are only required when the paginationType is of type URL. In all other scenarios, the action is executed normally.
Now, regardless of the route with which an organization is created (invitation, signup or self-create by user), the correct groups and permissions are assigned to the user.
We override the SimpleReactiveMongoRepository with our custom implementation to add criteria for filtering soft deleted records.
Also, adding a new function to archive record instead of a hard delete.
/actions should return all the actions of the current organization only.
Added organizationId to Action. This gets set only during create operation. Whenever get is performed on actions, it is filtered using organizationId.
See merge request theappsmith/internal-tools-server!164
At the moment, reactive Spring does not support AuditorAware implementation which also inserts the username of the user making the changes. Reference: https://jira.spring.io/browse/DATACMNS-1231.
This is to ensure that when we deploy in production, we can add redirect rules on Nginx. Now /api and /oauth2 urls will proxy to the API server while all other URLs will proxy to the frontend server.
Cleaning up the Github OAuth2 login flow by adding userNameAttribute to properties
This property helps define for Spring security which field in the OAuth2 user info to read in order to determine the username of the user. This is because this field is non-standard across different OAuth2 implementations. For each new OAuth2 provider that we support, this field will be required. Else the default name field will be picked up by Spring security (which is usually the id of the user).
See merge request theappsmith/internal-tools-server!156
This property helps define for Spring security which field in the OAuth2 user info to read in order to determine the username of the user. This is because this field is non-standard across different OAuth2 implementations. For each new OAuth2 provider that we support, this field will be required. Else the default name field will be picked up by Spring security (which is usually the id of the user).
The domain restriction has been done by adding parameter `hd` in the function CustomServerOAuth2AuthorizationRequestResolver#authorizationRequest. We still verify if the OAuth2 response has the parameter `hd` to ensure that no client side manipulation has been performed.
This property helps the spring security library derive the host name, protocol and port accurately even while running behind a Nginx load balancer. This is because nginx adds X-Forward-* headers that are parsed by Spring security library.
This is required for OAuth2 failure handling. When the client makes an OAuth2 request, we don't receive an origin header. If the OAuth2 login fails for some reason, our failure handler will not redirect correctly unless we also check for the referer header.
This is required because when we host inside a docker container, the default host and port picked up by the code is the docker container's name & port. This will not work when Google (for example) needs to redirect back to our server after authentication is complete. Hence, we need to customize the default redirect uri for all OAuth2 endpoints.
The issue was that the chain would receive a mono instead of an object. This would cause it to not persist the token in the DB. Moving to flatMap fixes this problem as we then deal with the actual object instead of the Mono.
Also fixing the url encoding in the email to ensure that the links generated are correct. By default mustache does HTML escaping and not URL encoding.
Also refactoring the code to use the MultiValueMap for filter params which is a generic implementation and can support filtering on multiple fields of the domain object groups. This can be replicated to other domain objects as well.
Ensuring that a user's personal workspace name is their first name by default.
This is because the full name makes the entire workspace name very long. If we just use the first name, it'll be a shorter string and easier to read by the end user as well.
See merge request theappsmith/internal-tools-server!142
This is because the full name makes the entire workspace name very long. If we just use the first name, it'll be a shorter string and easier to read by the end user as well.
2. Only dependencies of actions over other actions are counted. If the action is dependent on widgets, thats is not counted as a dependency and is left for the frontend to handle.
We use the referer header to find the client's endpoint and redirect to that endpoint once the OAuth2 login is successful. This is implemented via CustomServerOAuth2AuthorizationRequestResolver and AuthenticationSuccessHandler
Responding with 401 unauthorized http response header for all unauthenticated requests
This ensures that the client can redirect to the login page whenever it receives the http status 401 in the response header. Now, the server is no longer responsible for redirecting to the login page. All server redirects have been disabled.
See merge request theappsmith/internal-tools-server!128
This ensures that the client can redirect to the login page whenever it receives the http status 401 in the response header. Now, the server is no longer responsible for redirecting to the login page. All server redirects have been disabled.
Adding the invite user flow to invite a new user into the organization.
This includes multiple APIs:
1. Invite user which creates an invite token for the user and then sends it to the new user via email. This token is valid for 1 hour (for now).
2. An endpoint to verify the invite token so that the client can pre-emptively show any errors or expired token messages. This endpoint is publicly accessible and doesn't require a session.
3. An endpoint for the new user to sign up and set their password by providing their invite token. This endpoint is publicly accessible and doesn't require a session.
Also extending the redis session timeout to 1 day instead of 30 mins.
See merge request theappsmith/internal-tools-server!125
Handling duplicate email in forgotPassword flow.
Now if the user requests for the reset password token multiple times, we will override the previous token and save the new token against that user.
See merge request theappsmith/internal-tools-server!122
Correcting the HTTP methods in SecurityConfig and acl.rego to fix bug in forgotPassword and resetPassword authorization.
See merge request theappsmith/internal-tools-server!121
Adding a flag to determine if we should send emails for that particular environment
This is to prevent us from sending erroneous emails from our dev/test environments
See merge request theappsmith/internal-tools-server!120
Removing the logic to bind actions to pages when a dynamic binding happens.
This is because when an action is created, it is already bound to that page. This assumption will not change. Also, adding the pageId when searching for an action because the action name and pageId is a unique index. There may be actions with similar names across multiple pages.
See merge request theappsmith/internal-tools-server!119
This is because when an action is created, it is already bound to that page. This assumption will not change. Also, adding the pageId when searching for an action because the action name and pageId is a unique index. There may be actions with similar names across multiple pages.
OPA controls access to all endpoints and the list of authenticated resources and public URLs is defined in a single place in that file.
The url_allow function in acl.rego is an overloaded function that replicates the OR condition in Rego. Either the user is authenticated and has permissions to access those resources, or the URL is public and accessible by any user.
Now, we have an authenticationSuccessHandler & authenticationFailureHandler for OAuth & Form sign ups. This makes the whole flow much easier to handle.
Also redirecting the browser to this user profile endpoint on username password login success. This ensures that the client fetches the profile along with setting the session ID cookie in the browser
This involves overriding the default success and failure handlers for login in the SecurityConfig class. The handlers are defined in the filters package.
The default groups for an organization are created by copying over data from the groups collection with the organizationId 'default-org'. This ensures that we can update the permissions in the DB and add new permissions without any code changes. The new user is then assigned to these groups so that they have all the permissions required to administer the organization.
Also adding a display name for the groups so that the UI can show a different name than the one used internally within the DB to represent the group.
Checking if the name of the user is not null. Only then inserting it that field into analytics service.
See merge request theappsmith/internal-tools-server!97
Adding the base uri for google oauth2 redirect uri
This is required because when we host the server in a docker container, by default the baseUri picked up by Spring security is the DNS name of the appsmith server inside the docker networking bridge. These names may not be valid DNS names that Google can redirect to in the event of a successful Oauth2 login. Hence, we are overriding the base uri with our own uri for each environment.
The customer will have to provide this uri when they host it on their internal networks. This uri must be publicly accessible for Oauth2 to work.
See merge request theappsmith/internal-tools-server!90
This is required because when we host the server in a docker container, by default the baseUri picked up by Spring security is the DNS name of the appsmith server inside the docker networking bridge. These names may not be valid DNS names that Google can redirect to in the event of a successful Oauth2 login. Hence, we are overriding the base uri with our own uri for each environment.
The customer will have to provide this uri when they host it on their internal networks. This uri must be publicly accessible for Oauth2 to work.
Instead of sending complete application information, we now only send the boolean true or false value to indicate to the client the success or failure of the publish action
This is to ensure that the Action object is independent by itself and the client can derive the mustache keys by looking at the action object directly.
When we shut down the server, the redis subscription is not closed cleanly leading to the server to hang. Adding an error handler solves for this issue.
Note that we have to delete the Mongo volume for the inidtb script to take effect. Else, it will not execute. To be used only for testing right now. Will figure out a more robust solution later.
Also using Google's JIB Maven plugin to reduce our Docker image footprint. Will make container upload and download much faster
Open Policy Agent requires a bundle in the form of tar.gz in order to bootstrap itself with base policy and data. The server will serve this policy under the public domain. In the future, we will enable a Basic Authentication scheme in order to lock down this pocliy.
Currently, ACLFilter communicates with the OPA daemon to determine if the request should be validated or not based on the resource and permissions in user and group
This also fixes the logic for multiple ResponseDTOs that were being sent when the service level response was a Flux. Now we only return a single array inside a Mono ResponseDTO object.
We are moving to a multi-module structure so that different parts of the codebase can be exposed to the public while others can remain private. Using pf4j for plugin framework.
Also adding a build script `build.sh` which compiles the code and creates the `dist` folder for distribution purposes. Now we can build the code via
```
$ ./build.sh -DskipTests
```
This is not the prettiest way to do it. Ideally, we should send a query parameter to Google with key "hd=example.com" in order to get Google to list accounts with only those domains. For now, we send an error when the domains don't match in a custom `ClientUserRepository` function.
Now when we want to send any error message to the client, we should add it to the enum `AppsmithError` and throw an `AppsmithException` by passing this enum value to the constructor. In the future, we can also localize these error messages by defining the template message in a `messages_en.properties` file.
This ensures that we can send header values and those will be printed in the logs. In the class MDCFilter, we are adding the logContext to the reactive logger along with the normal MDC logger as well.
Also, introduced the util cascadeSave. This is done with the aim of storing the lower order documents first before storing the higher order composite documents (which contain DBRef).
Currently, there is a single executor implemented PostgresDBPluginExecutor which returns some dummy results. The entire framework exists to choose and instantiate the appropriate executor implementation at runtime.
This allows mux to compare and match the requests based on the HTTP method as well. If we compare the methods in the middleware chain, then mux will try to redirect the request to the first match it finds.
Other changes include:
* Also removing httprouter mux in favour of gorilla for being more mature and having more integrations and resources available for debugging.
* Adding http middlewares for logging req processing time and handling authentication.
TODO: Need to add context in the middleware as well. Will be useful for logging and debugging.
The config file `config.yml` must be in the same location as the binary. It must be named `config.yml`. For different environments, symlink the configuration file to the environment speciifc file.
