Commit Graph

1069 Commits

Author SHA1 Message Date
Arpit Mohan
aac31235f6 Merge branch 'master' of gitlab.com:theappsmith/internal-tools-server
# Conflicts:
#	appsmith-plugins/restApiPlugin/src/main/java/com/external/plugins/RestApiPlugin.java
#	appsmith-server/src/main/java/com/appsmith/server/services/ResourceContextServiceImpl.java
2019-11-07 14:28:09 +05:30
Trisha Anand
63c0f07708 Query param handling has been added to Rest API plugin using URIComponentsBuilder 2019-11-06 12:30:56 +00:00
Arpit Mohan
bcda3f0feb Renaming all the references of resource to datasource
This renaming is reflective of the entity and it's potential values in the future.
2019-11-06 16:25:08 +05:30
Arpit Mohan
a61b9ab325 Handling null values in the request body 2019-11-05 12:25:08 +00:00
Trisha Anand
3c1b08d911 Page Controller now can return list of pages by application id and list of page names & ids by application id 2019-11-05 11:49:52 +00:00
Trisha Anand
3f968afac6 Page Controller now can return list of pages by application id and list of page names & ids by application id 2019-11-05 11:36:10 +00:00
Arpit Mohan
0c0956ca46 Adding ACL bundle for datasources API 2019-11-05 16:36:09 +05:30
Arpit Mohan
3622d8b0d3 Converting actionConfiguration to a string to extract mustache keys
Also making jsonPathKeys a read-only field. Cannot be written via the API.
2019-11-05 10:40:09 +00:00
Trisha Anand
00fa2e80f7 Added ActionCollectionService which would act on composite functions. Need to handle all scenarios here. 2019-11-05 09:39:54 +00:00
Trisha Anand
bd846bfc8d Fix for dsl getting rewritten 2019-11-05 07:37:39 +00:00
Trisha Anand
68e30ff297 Handles empty params in a request 2019-11-05 05:51:14 +00:00
Trisha Anand
daeba007a1 Bug/rest api 2019-11-04 11:20:18 +00:00
Arpit Mohan
023ef93969 Fixing Heroku credentials again. Need to find permanent solution to this. 2019-11-01 08:52:20 +01:00
Arpit Mohan
00cd806259 Changing the Redis credentials because Heroku changed them again 2019-10-28 13:15:02 +01:00
Arpit Mohan
d17927e1f8 Reducing data response for application publish API
Instead of sending complete application information, we now only send the boolean true or false value to indicate to the client the success or failure of the publish action
2019-10-24 15:41:38 +05:30
Arpit Mohan
2bd8113229 Moving JsonPathKeys from Page document object to Action
This is to ensure that the Action object is independent by itself and the client can derive the mustache keys by looking at the action object directly.
2019-10-24 12:38:52 +05:30
Arpit Mohan
314cdd29d9 Handling error in RedisListenerConfig
When we shut down the server, the redis subscription is not closed cleanly leading to the server to hang. Adding an error handler solves for this issue.
2019-10-23 05:20:44 +00:00
Trisha Anand
d178b1c729 Publishing an application has been implemented by introducting publishedDsl field inside Layout. On Publish, for all pages in the application, for all layouts inside each page, the dsl json object is copied into the publishedDsl json object. 2019-10-23 04:41:51 +00:00
Trisha Anand
f8dddb212f Action Service - Delete 2019-10-21 14:01:21 +00:00
Trisha Anand
8ac0203b83 Fixing stupid bug in the acl rego file 2019-10-21 18:39:41 +05:30
Arpit Mohan
6eb7311197 Adding the permissions for fetching and updating applications & groups.
Also adding permission for deleting actions
2019-10-21 12:57:36 +05:30
Arpit Mohan
fc1a275670 Merge branch 'master' into hotfix/java-oom-fix 2019-10-21 12:25:05 +05:30
Arpit Mohan
5c1ef9f98a Adding the JVM parameter to ensure that java stays within the memory limits set by the container 2019-10-21 12:23:00 +05:30
Trisha Anand
74a8466528 Analytics Service now caters to all the events that need to be published. Every service should now call the analytics service's sendEvent function for tracking events. 2019-10-18 13:45:18 +00:00
Trisha Anand
84837fe57f Mongo Plugin works here. Barring the scenarios where the query json contains special characters in keys ($, .) it works properly. 2019-10-18 08:27:19 +00:00
Arpit Mohan
0473e3d48e Removing Heroku API key from gitlab-ci file 2019-10-17 12:44:37 +05:30
Arpit Mohan
04a8467095 Fixing the docker credentials in maven so that we don't require the docker daemon for CI/CD 2019-10-17 11:55:55 +05:30
Arpit Mohan
9550f6210d Removing image name from the package step in gitlab-ci file 2019-10-17 11:39:37 +05:30
Arpit Mohan
75a5e48c4d Adding docker hub access token to gitlab-ci for container upload 2019-10-17 09:47:23 +05:30
Arpit Mohan
3a9f344a51 Removing plugins from appsmith-interfaces module 2019-10-17 09:25:58 +05:30
Arpit Mohan
d295875ee3 Removing the git-commit plugin because it doesn't run properly on heroku 2019-10-17 09:18:17 +05:30
Arpit Mohan
135b61a4bc Adding mongo seed data that will be recreated whenever docker-compose causes the containers to restart
Note that we have to delete the Mongo volume for the inidtb script to take effect. Else, it will not execute. To be used only for testing right now. Will figure out a more robust solution later.

Also using Google's JIB Maven plugin to reduce our Docker image footprint. Will make container upload and download much faster
2019-10-16 11:27:02 +00:00
Arpit Mohan
f51de50725 Adding more acl rules for properties and action controllers 2019-10-14 12:29:20 +00:00
Arpit Mohan
cf5b2c325b Adding a folder for public resources in order to serve OPA bundle
Open Policy Agent requires a bundle in the form of tar.gz in order to bootstrap itself with base policy and data. The server will serve this policy under the public domain. In the future, we will enable a Basic Authentication scheme in order to lock down this pocliy.
2019-10-14 09:24:09 +00:00
Trisha Anand
e1d59ca3a6 Multi data source support for plugins using Resource Context 2019-10-11 10:32:18 +00:00
Arpit Mohan
632aff4faa Adding basic ACL functionality with Open Policy Agent.
Currently, ACLFilter communicates with the OPA daemon to determine if the request should be validated or not based on the resource and permissions in user and group
2019-10-11 10:31:28 +00:00
Trisha Anand
7315802508 Fixing redis connection changes 2019-10-10 11:53:03 +00:00
Trisha Anand
d1bcc282f8 Adding the redis listener via spring-data-redis-reactive.
The listeners need to be configured in the RedisConfig class via Beans. These beans can then invoke complex business logic based on requirements.
2019-10-07 09:02:02 +00:00
Arpit Mohan
ad22e1a406 Adding the redis listener via spring-data-redis-reactive.
The listeners need to be configured in the RedisConfig class via Beans. These beans can then invoke complex business logic based on requirements.
2019-10-07 08:18:54 +00:00
Trisha Anand
59752254b4 On install, the plugins are loaded and started again. This ensures that we don't have to restart the server every time a new plugin is installed. Earlier the plugins were loaded and started only during the boot 2019-10-04 04:27:11 +00:00
Trisha Anand
8658df95a9 All action executions now return object of type ActionExecutionResult. RestApiPlugin returns the same object which contains statusCode, headers and body. 2019-09-30 18:17:35 +00:00
Arpit Mohan
71ac6f4488 Updating to the latest spring boot version 2019-09-30 11:32:16 +05:30
Trisha Anand
51d5c93119 Fixing minor bugs in the Segment analytics flow 2019-09-25 17:06:28 +00:00
Trisha Anand
566a245451 Segment & Rollbar Integration 2019-09-25 16:20:51 +00:00
Trisha Anand
bec8b2e0a1 Adding the docker-compose up command to the build
This should ideally run the mongo-seed container to restore the DB before each run
2019-09-24 12:14:41 +00:00
Trisha Anand
de55ec8f8e Property Pane for Widget 2019-09-24 11:40:53 +00:00
Arpit Mohan
280f8d4dcb Fixing the Cors configuration to ensure that pre-flight requests return the Access-Control-Allow-Origin header 2019-09-18 09:45:00 +00:00
Arpit Mohan
ac44e2c970 Modifying the ResponseDTO to contain the ResponseMeta object. Also moving ErrorDTO inside ResponseMeta object.
This also fixes the logic for multiple ResponseDTOs that were being sent when the service level response was a Flux. Now we only return a single array inside a Mono ResponseDTO object.
2019-09-18 08:52:14 +00:00
Trisha Anand
3d82809d2a CI/CD with Heroku and docker via gitlab 2019-09-18 08:24:03 +00:00
Arpit Mohan
6f4bfdd728 Ensuring that the plugins directory is in the same path as the jar 2019-09-17 18:02:58 +05:30
Arpit Mohan
b618bfc5d8 Adding a very initial version of the RestApiPlugin
Requires a bunch of testing for corner cases and error scenarios.
2019-09-17 12:24:45 +00:00
Arpit Mohan
0edc0f6363 Adding the executeAction API in the ActionController.
Also changed the interface for the PluginExecutor to include the resourceConfig and ActionConfig as well.
2019-09-17 12:18:23 +00:00
Trisha Anand
4498a301a6 Docker 2019-09-17 09:35:13 +00:00
Trisha Anand
14ec5260d9 Incorporated review comments from Nikhil on action 2019-09-17 07:23:30 +00:00
Trisha Anand
5468436d9f Add Page Actions to Page object. 2019-09-17 06:25:07 +00:00
Arpit Mohan
f5ea85f636 Running only part of build.sh in the buildpack-run.sh. This is because all the buildpacks in Heroku are run sequentially 2019-09-16 17:19:55 +05:30
Arpit Mohan
cdba3a5b68 Adding Procfile for custom run command in Heroku 2019-09-16 17:12:26 +05:30
Arpit Mohan
bb84e74c23 Re-arranging the files to fix bad merge that threw compile errors 2019-09-16 17:04:07 +05:30
Arpit Mohan
661111daaa Setting java version to 11 in system.properties.
This is required by Java Heroku buildpack
2019-09-16 16:54:28 +05:30
Arpit Mohan
b3c987a9f9 Convert the project into a multi-module structure
We are moving to a multi-module structure so that different parts of the codebase can be exposed to the public while others can remain private. Using pf4j for plugin framework.

Also adding a build script `build.sh` which compiles the code and creates the `dist` folder for distribution purposes. Now we can build the code via

```
$ ./build.sh -DskipTests
```
2019-09-16 11:21:57 +00:00
Trisha Anand
467b3e6db5 Fixes nested data object in Layout. Renamed data to 'dsl' 2019-09-11 11:51:25 +00:00
Trisha Anand
07614d8980 CRUD for pages and layouts. 2019-09-11 10:44:31 +00:00
Trisha Anand
7e0cb75ed2 Added domains for application and pages. Edited the layout domain to make it an embedded object inside a page instead of existing in the database on its own. 2019-09-09 06:54:36 +00:00
Arpit Mohan
9c47e1cb2f A hacky approach to limiting the Google OAuth2 to a set of custom domains.
This is not the prettiest way to do it. Ideally, we should send a query parameter to Google with key "hd=example.com" in order to get Google to list accounts with only those domains. For now, we send an error when the domains don't match in a custom `ClientUserRepository` function.
2019-09-09 05:16:42 +00:00
Trisha Anand
bea5f53f46 Signup flow 2019-09-06 10:49:48 +00:00
Arpit Mohan
f71bd9a4e0 Migrating from Tenant nomenclature to Organization.
This makes it consistent with the terminology that we want in the product. The hierarchy will be organization -> team -> user.
2019-09-04 15:08:40 +05:30
Arpit Mohan
bf908025e2 Adding some basic unit tests for the TenantService class. We should add more tests for other CRUD flows as well. 2019-09-04 09:24:38 +00:00
Trisha Anand
4258f85a81 Removed DBRef from action. Added DBRef of plugin because while execution, this will be used. So instead of fetching plugin on every execution, plugin is fetched once during create action and stored as part of action (DBRef). Because of DBRef, eager loading would happen which would lead to plugin being present on get without loading plugin again. 2019-09-03 11:36:49 +00:00
Trisha Anand
7bc8948116 Correcting tenant id on staging 2019-08-29 15:39:42 +05:30
Arpit Mohan
bdc53195f0 Sending standardized HTTP error codes to the client application. This structure ensures that we can define the http error code, app error code and (in future) localized string when we want to.
Now when we want to send any error message to the client, we should add it to the enum `AppsmithError` and throw an `AppsmithException` by passing this enum value to the constructor. In the future, we can also localize these error messages by defining the template message in a `messages_en.properties` file.
2019-08-28 09:35:06 +00:00
Trisha Anand
bbbdfbada3 Action CRUD implemented 2019-08-27 11:31:44 +00:00
Arpit Mohan
06945ff10f Renaming Mobtools to Appsmith everywhere for consistency 2019-08-27 15:14:50 +05:30
Trisha Anand
43780a6550 Action configuration now contains all the configurations. The global and mustache configurations now exist under the same object. Data template object has been removed. This is inline with the design discussion. 2019-08-27 09:37:38 +00:00
Arpit Mohan
5a1f9ab132 Adding MDC logging via RequestIdFilter and MDCFilter
This ensures that we can send header values and those will be printed in the logs. In the class MDCFilter, we are adding the logContext to the reactive logger along with the normal MDC logger as well.
2019-08-27 06:25:29 +00:00
Trisha Anand
0449633997 Removed wild card imports. Now wild card imports would only happen if the import class count exceeds 20. 2019-08-22 12:03:42 +00:00
Trisha Anand
e852ec6c15 The create tenant flow is completely reactive. This was done because the earstwhile implementation was erroneous leading to unpredictable crashes. 2019-08-12 04:44:28 +00:00
Trisha Anand
ae0fd72d48 The create tenant flow is completely reactive. This was done because the earstwhile implementation was erroneous leading to unpredictable crashes. 2019-07-28 04:46:53 +00:00
Trisha Anand
68e1bef0f6 Introduced TenantSetting to store the real setting value. Setting would exist as set of settings that can be configured for a given tenant. The TenantSetting document stores the configured values for each setting.
Also, introduced the util cascadeSave. This is done with the aim of storing the lower order documents first before storing the higher order composite documents (which contain DBRef).
2019-07-25 07:11:05 +00:00
Arpit Mohan
2a5656afbb Changing the query execution to execute by name. Will make the API a lot simpler. 2019-04-02 21:55:19 +05:30
Arpit Mohan
bbb6e511d7 Adding a working version of the rest template plugin executor. 2019-04-02 21:10:36 +05:30
Arpit Mohan
6d91ab32ce Adding the bean corsWebFilter to enable CORS in our codebase 2019-04-02 20:05:48 +05:30
Arpit Mohan
c14b34193d Refactoring the plugin implementations to have their own package.
Will keep the code more organized
2019-03-31 21:55:27 +05:30
Arpit Mohan
5570020f6d Adding Configuration annotation on CORS config class 2019-03-31 21:49:39 +05:30
Arpit Mohan
cffb83a781 Disabling CORS by effectively allowing requests to be made from all hosts 2019-03-31 21:48:50 +05:30
Arpit Mohan
303f42945c Correcting the properties reading from PostgresDBPluginExecutor
Don't use static variables if using spring to feed the config values in
2019-03-31 21:42:29 +05:30
Arpit Mohan
a82a931093 Adding mustache template substitution to QueryObj.
Now we can query postgres DB on the fly! Yipee!
2019-03-31 21:22:06 +05:30
Arpit Mohan
28ef85b893 Adding name and confirmationMsg to the query collection 2019-03-30 16:41:44 +05:30
Arpit Mohan
ad2310bae0 Adding properties to a widget 2019-03-30 16:22:14 +05:30
Arpit Mohan
3bdfd13284 Adding the APIs to add plugins and query commands
Currently, there is a single executor implemented PostgresDBPluginExecutor which returns some dummy results. The entire framework exists to choose and instantiate the appropriate executor implementation at runtime.
2019-03-30 16:14:54 +05:30
Arpit Mohan
a57d31fbb2 Adding the system.properties file for java version
Heroku needs this for determining java version
2019-03-30 13:12:49 +05:30
Arpit Mohan
8f4bfd59fc Adding application-local.properties
This is to ensure that we can provide different types of mongo configurations for different environments
2019-03-30 13:08:43 +05:30
Arpit Mohan
e553470095 Moving the code to the top level to remove one-level of folder indirection 2019-03-30 13:04:05 +05:30
Arpit Mohan
4733ebf95b Removing Go code from the repo. This is now pure Java spring boot app 2019-03-30 13:01:06 +05:30
Arpit Mohan
d786890834 Adding the staging properties with mongodb atlas setup 2019-03-30 12:58:34 +05:30
Arpit Mohan
cef6946859 Adding the LayoutController that will store the DDL for the page layouts
Also adding the getById function in the BaseController so that all CRUD APIs will automatically support that API call.
2019-03-30 12:23:21 +05:30
Arpit Mohan
f8b382940d Adding basic authentication to the server.
This will allow us to host it on a webserver for integration purposes.
2019-03-30 12:01:24 +05:30
Arpit Mohan
8ccd001652 Adding abstractions for CRUD APIs.
Now we can easily add controllers for any resource quickly by simply adding a few files.
2019-03-19 19:39:05 +05:30
Arpit Mohan
f7aaafacfb Moving to Mongo instead of Postgresql for storage.
This will allow us to store the widget, plugin & page information much more cleanly and directly into the DB.
2019-03-19 12:13:21 +05:30
Arpit Mohan
9cea9f94f2 Completing the CRUD for the widget domain. 2019-03-19 02:28:00 +05:30
Arpit Mohan
e4f59ff941 Adding the Spring java API for creating Widget.
Migrating the code to Spring Java instead of Golang
2019-03-19 01:50:32 +05:30
Arpit Mohan
7ea78bf64c Adding functionality to execute a query via mustache template. 2019-03-16 18:28:44 +05:30
Arpit Mohan
49b7051cea Moving the method comparison in gorilla mux to the end and not in the chain
This allows mux to compare and match the requests based on the HTTP method as well. If we compare the methods in the middleware chain, then mux will try to redirect the request to the first match it finds.
2019-03-16 17:05:20 +05:30
Arpit Mohan
1133b53437 Adding Google authentication via Goth. All endpoints can now be authenticated.
Other changes include:
* Also removing httprouter mux in favour of gorilla for being more mature and having more integrations and resources available for debugging.
* Adding http middlewares for logging req processing time and handling authentication.

TODO: Need to add context in the middleware as well. Will be useful for logging and debugging.
2019-03-16 15:47:47 +05:30
Arpit Mohan
edaf7e3025 Correcting the project structure as per idiomatic go 2019-03-10 15:51:19 +05:30
Arpit Mohan
ce34ff4a83 Completing the CRUD for the query API. Also being able to execute a saved query.
TODO: Handle parameters while executing a saved query.
2019-02-28 09:41:35 +05:30
Arpit Mohan
b38df1c07d Minor refactoring 2019-02-28 00:46:17 +05:30
Arpit Mohan
dc66c070d6 Adding viper configuration to make the server configurable.
The config file `config.yml` must  be in the same location as the binary. It must be named `config.yml`. For different environments, symlink the configuration file to the environment speciifc file.
2019-02-28 00:35:50 +05:30
Arpit Mohan
3cfadc1c51 Completing the CRUD API for component model. Also adding more basic structs for other models 2019-02-27 22:05:03 +05:30
Arpit Mohan
eb2af8e534 Minor comments 2019-02-27 09:44:02 +05:30
Arpit Mohan
13a0247fd7 Creating a generic map to struct function to convert any map objects into struct objects
Also creating a services package. This will hold all the business logic. The controllers will be very light.
2019-02-27 09:29:30 +05:30
Arpit Mohan
4016562f09 Making the ExecuteQuery function generic to return a map of rows instead of a struct
This will help us to execute arbitrary queries and map them to results at a later stage.
2019-02-26 23:49:20 +05:30
Arpit Mohan
acbb5a55e0 Moving gorm ORM library for Golang database connections
Also creating the Create API for components
2019-02-26 23:22:37 +05:30
Arpit Mohan
13156380ce Adding the factory pattern to switch between databases.
Also adding a basic GET API for components

Making modifications to the Makefile to create a dist/ folder.
2019-02-26 22:36:21 +05:30
Arpit Mohan
f72597051c Adding the factory method to derive postgres implementation for db 2019-02-22 10:16:46 +05:30
Arpit Mohan
6560d1005c Adding the files for the APIs 2019-02-22 08:53:03 +05:30
Arpit Mohan
2be56a1f12 Adding httprouter as the HTTP multiplexer and also setting up postgres db connection and executing a random query
This basically sets up some rough structure of the code
2019-02-22 08:48:08 +05:30
Arpit Mohan
21de87596c Adding the go module init file go.mod 2019-02-21 23:21:07 +05:30
Arpit Mohan
82289b5041 Initial commit. Adding the Makefile and hello world code 2019-02-21 10:08:45 +05:30
Arpit Mohan
53ae0ba4e5 Initial commit 2019-02-17 05:41:20 +00:00