## Problem
**Why do we need validation dependency?**
- When on change of particular property value if other property needs revalidation then current evaluation architecture do not support such validation run.
https://www.notion.so/appsmith/Validation-Dependency-d7623a5625bd4aa187a3ae2372d3ac07
**Issues to resolve**
Fixes#15303Fixes#17159
Fixes https://github.com/appsmithorg/appsmith/issues/16170
## Solution
**New Validation Flow**
validate a property after its evaluation and check if there are other properties that need to revalidate on a change of this value.
- if yes run validation for those properties.
How does validation dependency get created?
- every widget’s property will keep static `dependencies` array in `validation` property like shown below.
```jsx
{
validation: {
type: ValidationTypes.FUNCTION,
params: {
fn: defaultOptionValueValidation,
expected: {
type: 'value1 or { "label": "label1", "value": "value1" }',
example: `value1 | { "label": "label1", "value": "value1" }`,
autocompleteDataType: AutocompleteDataType.STRING,
},
dependentPaths: ["serverSideFiltering", "options"],
},
dependencies: ["serverSideFiltering", "options"],
}
}
```
This `dependentPaths` array will be used to form validationDependencyMap.
#### Changes
- The code editor component wasn't re-rendering for a few cases. Due to this updates were not shown until the component state was changed.
- In CreateFirstTree, After evaluateTree we run `validateTree`. In `validateTree`, we validate all the properties in `validationPaths`. Here, if the errors were resolved on revalidation, it didn't reset the validation errors stored in data tree before. Now, this PR adds the reset logic.
- Created `validationDependencyMap` to re-validate properties on change of the property their validation depends upon.
- After each property evaluates in `updateDataTree`, we check if re-validation is needed, if yes then we re-validate dependent properties.
## Type of change
- Bug fix (non-breaking change which fixes an issue)
- New feature (non-breaking change which adds functionality)
## How Has This Been Tested?
- jest test
**Test Plan**
https://github.com/appsmithorg/TestSmith/issues/2078
## Checklist:
- [x] My code follows the style guidelines of this project
- [ ] I have performed a self-review of my own code
- [ ] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my feature works
- [x] New and existing unit tests pass locally with my changes
* update rts logic to use updated shared AST logic
* Make changes to naming conventions
* Add test cases for RTS and rename ast functions
* Add running jest test to RTS workflow
* Install dependencies and then trigger jest tests in workflow
* Close server connection after test ends
* Remove logs
* Improve jest test descriptions
* fix: updated skip functionality
* update: type of logs to add occurence count
* update: reducer function to merge the new logs
* update: added dependance on last log's occurence count
* add: UI to show the occurence badge
* fix: added null check for dependency array
* update: omit occurence from compare fn
* update: changed function call to store logs array directly
* update: moved from saving logs one by one to array
* fix: replaced forEach with reduce
* test: added functions for console log grouping
* feat: updated warning grouping color
* update: moved function to reducer file for jest tests
* Initial commit
* Remove arrow function params from identifiers
* Remove invalid identifiers from extracted identifiers
* Remove invalid identifiers which are derived from function params and variable declarations
* Fix typo error
* Correctly remove invalid identifiers
* Remove invalid names from identifier list
* fix build failure
* Add Promise to list of unacceptable entity name
* Keep track of unreferenced identifiers in bindings
* Add Global scope object names as unusable entity names
* Keep track of unreferenced identifiers
* Prevent traversal of data tree for addition of new paths and entities
* Sync linting in trigger fields
* Support linting of invalid properties
* Fix linting reactivity bug in trigger field
* Remove unused objects
* Fix conflict in merging
* Lint jsobject body for function change
* Remove unused map from tests
* Code cleanup
* Modify jest tests
* Update jest tests
* Fix cypress tests
* Code cleanup
* Support linting of multiple bindings
* Set squiggle line as long as invalid property length
* Add jest tests
* Minor code refactor
* Move ast to shared repo
* Rename confusing identifiers
* Improve naming of functions and their return values
* move shared widget validation utils and constants to shared folder
* Add jest test for invalid entity names
* Add cypress tests
* Modify test comment
* Extend list of dedicated worker scope identifiers
* Resolve code review comments
* Resolve review comments
* Annonate code where necessary
* Code refactor
* Improve worker global scope object
* Code refactor
* Fix merge conflict
* Code refactor
* Minor bug fix
* Redundant commit to retrigger vercel build
* Add null checks to dependecy chain
For nested functions, the trigger meta is not available for subsequent functions. So in cases where the dynamic trigger has the relevant info, we can use that to update the trigger meta and then use that meta to display the source of the logs.
Fixes#16515Fixes#16483Fixes#16514
Co-authored-by: Aishwarya UR <aishwarya@appsmith.com>
* feat: capture console from across the app (#15676)
* create: console override file
- Adds methods to override the given console functionality to capture the console statements written by the user
* update: evaluate function to extract console op
- Added logs extraction for both sync and async functions
- Adding them to the return object of the evaluations
* update: timestamp field to log addition method
- Added optional argument to function definition for inputting the timestamp of the log
- This is done to maintain timings of the execution of the log
* update: interface for log objects
* update: post function execution logic
- Added logic to push the logs generated by the evaluation to the logs store
* update: added handling for sending nested fns
- While console logging functions or objects that had functions was causing an error
- Added a check for removing functions and replacing them with name of the functions instead
* chore: added types and comments
* fix: updated evaluation tests
* fix: added check for log in returned obj
* update: added the source data in the trigger logs
- Removed on js execute logs from showing up here since they are already handled. If they are not removed, they will show up on the first page load twice
* add: ellipsis function for log title string
- This is to keep big object contained in the first line only
* update: made logs reset function public
* update: resetting logs before new eval
- Logs object has to be cleared before next eval can happen to make sure there are no roll overs from last evals
* chore: added comments
* add: extracting logs after eval of functions
* add: storing logs to redux after eval
* refactor: updated types
* add: func to store logs w/ severity as arg
* refactor: updating func call for user logs
* chore: fixed elipsis logic
* chore: removed unused type
* chore: updated preview text logic
* add: type for transfer object post eval
* update: aded new userLogs obj to dataTreeEvaluator
* update: passing logs from object to saga
* update: parsing received userlogs
* refactor: used predefined fns
* refactor: moved resetlogs to common func
* chore: updated comments
* feat: update redux store and UI for system + user logs (#15936)
* update: updated types for the redux store
- Added category and data fields in the log object
* update: types of log redux store
* update: calls for the console log store function
* update: icon fetch func for log item UI
* update: syncing UI with the new designs (WIP)
* chore: fixed lint error
* update: filters for logs
* update: icon for clearing log filters
* update: filtering function
- Added checks against category and severity
* update: logitem UI
- updated type of the UI object
- added css based ellipsis
- added toggle for console logs
- added array of json views for objects/arrays
- css tweaks
* update: debugger cta
- Removed copy option
- Updated UI
* update: logic for expanding user logs
- Removed debugger CTA
- Fixed position for the expand/collapse icon
- Added joining char for when the log is expanded
* update: assets for new UI
- updated colors
- Added new icon
* hotfix: ternserver code
* add: search across the text of log
* update: icons for the app
* update: click to expand/collapse of logs
* fix: search keyword update on change within JSObjects
* fix: alignment of log items in both states
* update: jest tests for debugger errors and filter
* fix: drop down options color issue
- the icon used was not the standard one
- We have a lot of duplicates of the same icon
* fix: synced with ADS changes on release
* fix: remove dependency from old icon
* add: cypress selectors for automation testing
* fix: replaced static messages with variables
* fix: updated the dependency map for filter
* fix: height of the filter drop down
* fix: chaining logic for search filter
* fix: syncing the padding values to ADS
* fix: help icon visibility issue
* fix: width of filter dropdown
Co-authored-by: Rishabh-Rathod <rishabh.rathod@appsmith.com>
* add: e2e tests for console statements using IIFE
* test: added test for console logs in jsobjects
* fix: functionality of expanding context menu on msg click
* fix: added try catch and handling for numbers
* hotfix: handling unwanted toasts
* fix: alignment and clickable cursor
* fix: alignment of expanded span in console logs
* add: analytics event on new console log created
* fix: added handling for boolean and undefined
* fix: removed log reset from common func
- Whenever we are creating global DTO, the logs were being reset. This caused logs to reset whenever a promise was encountered in the logs.
* fix: combined JS log saving to widget process
* add: new analytics event on filter changed
* update: added handling for empty value
* update: removed comma between multiple logs
* update: synced test changes with release
* update: removed unused wait timings
* Logs spec script update
* update: ts methods in log spec e2e test
* logs spec update
* update: removed body clicks from test script
* Logs spec update
* update: removed ask from google option
* refactor: ui fixes
* fix: text selection of logs
* fix: updated dropdown width management
* update: made the flushlogs function async
* update: added handling for promises
* update: added test with promises fail and pass
* fix: added sync variant to work for sync objects
* refactor: commented out unused tests
* update: exceptions in the name of log entity
* fix: pagination of logs to handle dynamic data stream
* fix: removed unused async function
* fix: moved logs handling to separate saga
* fix: color for context menu text
Co-authored-by: Rishabh-Rathod <rishabh.rathod@appsmith.com>
Co-authored-by: Aishwarya UR <aishwarya@appsmith.com>
* refactored code for invite modal changes for rbac
* code splitted some more files for refactoring invite modal component
* removed unused imports
* created new variable for handlers
* updated an import
* reverted a change
* refactored a section of code
* fixed a cypress test
* fixed a cypress test
* updated imports
* exported some entities
## Description
This reverts commit dbcc643ccf.
Fixes#16378
## Type of change
- Revert changes which impacted performance
## How Has This Been Tested?
Manual performance verification
@ramsaptami
## Checklist:
- [ ] My code follows the style guidelines of this project
- [ ] I have performed a self-review of my own code
- [ ] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [ ] My changes generate no new warnings
- [ ] I have added tests that prove my fix is effective or that my feature works
- [ ] New and existing unit tests pass locally with my changes
Co-authored-by: ohansFavour <fohanekwu@gmail.com>
* POC for Shared AST Logic using Yarn Symlinks
* fix: preinstall script for bundling shared packages
* Merge commit
* fix: updated the script to link, unlink the package as shared dep
* fix: updated dependencies
* Add a post-install script and fix yarn.lock file
* Remove commented code
* fix: added verification script, readme, moved scripts to shared
* Extraction of AST Logic into shared/ast folder
* Add jest test script
* Replace hardcoded ast Logic use with Shared AST module
* Replace parse code with getAST
Co-authored-by: Aman Agarwal <aman@appsmith.com>
* POC: Datatype handling autogenerated naming params client
* added array and file datatype
* handles all primitive types
* paramproperties convetred to map instead of array
* parametermap inversion
* handled no bindings bug
* feat: Consume action execution payload changes w.r.t. data type mapping (#15555)
This commit does the following things
1. It consumes data type metadata from payload
2. It consumes the mapping between the pseudo binding name and original binding name
3. It doesn't apply any logic w.r.t. data type identification after consumption
* added comments and cleaned code with proper datatypes
* removed URLencoding for binding params names
* feat: Remove URL decoding on param keys from server-side codebase (#15555)
* Remove unused import (#15555)
Co-authored-by: ChandanBalajiBP <chandan@appsmith.com>
Add Autocomplete support for
- local variables
- JSObjects
- ButtonGroupWidget
Remove Autocomplete suggestion for
- `eval`
- undefined global values like `tabs`
* Remove Dropdown, SearchComponent from this repository
* Change the imports so that SearchComponent and DropdownV2 come from the design system
* Add yalc
* default imports become named imports
* remove the yalc package, add the beta version
* Change import for SearchComponent in TableHeader
* Updated Label Config
* chore: Refactor dependency Map code (#14944)
* feat: Triggerpath dependency (#15015)
* Introduce triggerpathdependency
* Code cleanup and refactor
* add jest test for removeLintErrorsFromEntityProperty
* Add triggerfieldDependencyMap jest tests for datatreeEvaluator
* Add cypress tests
* Revamp linting architecture
* Lint when creating first tree
* Prevent modification of unevaltree while linting
* improve linting performance
* Reuse allkeys for all dependencymap creation
* Add jest test for linttree
* Update jest and cypress tests
* Modify linting folder name
* remove redundant commits
* update evaluation jest tests
* Fix failing test
* Update cypress tests
* Code clean up and performance upgrade
* Update cypress tests
* Use unevalTree as global data for non-trigger field
* Improve global data creation time
* Cypress fixes
* Improve creation of global data for linting
* Only generate global data with functions when needed
* remove all unused variables
* Add overriden commit
* remove lint erros before linting
* fix merge conflicts
Co-authored-by: Aishwarya UR <aishwarya@appsmith.com>
Co-authored-by: Nikhil Nandagopal <nikhil.nandagopal@gmail.com>
Co-authored-by: Rishabh Rathod <rishabh.rathod@appsmith.com>
Co-authored-by: Favour Ohanekwu <fohanekwu@gmail.com>
Co-authored-by: Aishwarya UR <aishwarya@appsmith.com>
## Description
This PR adds [crypto web api](https://developer.mozilla.org/en-US/docs/Web/API/Crypto) support in linting
Fixes#15469
## Type of change
- Bug fix (non-breaking change which fixes an issue)
## How Has This Been Tested?
- [x] https://github.com/appsmithorg/TestSmith/issues/1996
## Checklist:
- [x] My code follows the style guidelines of this project
- [x] I have performed a self-review of my own code
- [x] I have commented on my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my feature works
- [x] New and existing unit tests pass locally with my changes
## Description
Previous version of the astring library doesn't preserve parenthesis when transforming AST to JS Code. This led to syntax error when null coalesce and logical expressions are used together.
The latest version of astring library fixes this.
<img width="1291" alt="screenshot_2022-07-16_at_04 29 52" src="https://user-images.githubusercontent.com/46670083/179581475-863d70f2-0886-403a-a667-a32ad93e76b7.png">
Fixes#15283
## Type of change
- Bug fix (non-breaking change which fixes an issue)
## How Has This Been Tested?
Test plan
- [x] https://github.com/appsmithorg/TestSmith/issues/1970
## Checklist:
- [x] My code follows the style guidelines of this project
- [x] I have performed a self-review of my own code
- [ ] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [ ] My changes generate no new warnings
- [ ] I have added tests that prove my fix is effective or that my feature works
- [ ] New and existing unit tests pass locally with my changes
* Introduce triggerpathdependency
* Code cleanup and refactor
* add jest test for removeLintErrorsFromEntityProperty
* Add triggerfieldDependencyMap jest tests for datatreeEvaluator
* Add cypress tests
* Revamp linting architecture
* Lint when creating first tree
* Prevent modification of unevaltree while linting
* improve linting performance
* Reuse allkeys for all dependencymap creation
* Add jest test for linttree
* Update jest and cypress tests
* Modify linting folder name
* remove redundant commits
* update evaluation jest tests
* Fix failing test
* Update cypress tests
* Code clean up and performance upgrade
* Update cypress tests
* Use unevalTree as global data for non-trigger field
* Improve global data creation time
* Cypress fixes
* Improve creation of global data for linting
* Only generate global data with functions when needed
* remove all unused variables
* Add overriden commit
* remove lint erros before linting
* fix merge conflicts
Co-authored-by: Aishwarya UR <aishwarya@appsmith.com>
Core changes in this PR
Prevents logging of lint warnings (not errors) in the debugger
Refined message for using 'await' in non-async functions
Show lint error messages that include line numbers (previously weren't shown)
Co-authored-by: Aishwarya UR <aishwarya@appsmith.com>
* fix: JsObject parsing
* fix jest test and add types
* remove logs from test
* Add jest test to verify this
* Add perf metric and
* fix: jsObject stale value
- refactor updateJSCollectionInDataTree
* Refactor JSObject code
* fix import path
* refactor and fix
* undo debugging change
* Add comments
* Add Ast for getting function params
* Fix and add test
* remove comment
* remove comment
* Resolve comments
* Client changes 1
* add DSL functionality
* Temp commit for refactoring changes
* Do I even know what I'm doing here?
* chore: Second GS layout
* Update: Visibility conditional outputs for schemas
- Added the output from conditional outputs for schema children too
* Update: Entity selector visibility control
- Added logic for controlling visibility of sub components via the JS expressions system
* Update: Passing disabled prop to toggle button
* Update: Passing disabled prop to toggle btn
* Update: Styled component for toggle button
- Added disabled styles based on the disabled prop sent to the toggle form view JSON button
* Update: configProperty role in Entity Selector
- Removed dependance of the configProperty of the entity selector children to it's parent component
* Update: type of placeholder key
- Made placeholder key from form config JSON to accept either string or an object
- Earlier only string was accepted
- This is for pagination component
* Update: Added placeholder control for pagination
* Client changes 1
* add DSL functionality
* Do I even know what I'm doing here?
* fix: updated uqi forms ui, clubbed JS switch button to ads, updated tooltip design
* fix: updated tooltip component for wrong ui on entity explore
* temp triggers
* fix: updated uqi forms ui, clubbed JS switch button to ads, updated tooltip design (#12395)
* fix: updated uqi forms ui, clubbed JS switch button to ads, updated tooltip design
* fix: updated tooltip component for wrong ui on entity explore
* fix: updated tooltip ui, where condition placement, sort by ui
* temp form data access logic
* fix: updated sorting type width ui
* fix: updated ui for spacing, width and text issues
* Update: Type for tooltip of UQI forms
- Added option to send an object to the tooltipText object.
- This allows for composite components like pagination to have tooltips for each sub component
* Update: tooltip for pagination component
- Added handling to parse the tooltip for multiple components.
- This allows for composite components like pagination to have tooltips for each sub component
* Update: Type cast for tooltip component
- Made the content passed to tooltip component as a string only
* Update: Fixed tooltip component CSS
* Update: Dropdown option component
- Added a tooltip wrapper to each option
- This is to show on hover text like disabled state
* fix: updated ẇhere clause broken ui for condition
* Add: functions to check and extract expressions
- Loop through the formConfig and find any keys that have a value that is bindable
- Used pre defined regex to check if value is a moustache binding
* Add: Types for evaluated form configs
- Added types for the form configs to be evaluated and their output post eval
* Add: Flow to run the form config
- Run the form config and update the result to the redux state
* Update: Name of the type for formconfigs
- Updated since it was clashing with a component of the same name
* Add: Function to enforce config type checks
- This is done so that the improper configs can be weeded out and the rest of the form can be shown
* Add: Function to update evaluated config
- Added option to update the config if it's values needed evaluation
* Add: Type check for schema sections
* Update: Error handling for invalid control type
- We were throwing an exception till now, changed it to a warning text
* Add: Exposed tooltip for dropdown option disabled state
* Update: switch to json mode functionality
- Added logic to convert data to a string rather than an object when the first switch to JSON mode happens
* Update: Added key to tooltip for dropdown options
* Trigger API modification
* Add: function to fetch default trigger URL
* Update: Made URL optional in dynamic trigger config
* Update: Dynamic trigger API call
- Made the API call for dynamic triggers have URL as optional field
- Added type check to the response of the API call
* Update: resp type for trigger APIs
* Update: Moved code to utils folder
- Moved functions for UQI form eval processing to utils file
* Update: passing original controltype to JS switch
* Update: config for JSON editor mode
- Updated the config to have different options for JSON mode depending on the original control type
* Update: Connected line numbers flag to config
* Revert: CSS changes for tooltip
* Refactor: Removed consle
* Add: type for the config of dynamic values
* Add: Feature to evaluate config for triggers
* Refactor: fix type check errors
* fix: dropdown ui width with text alignment
* Update: fixed selector for dynamic values
* Update: selector call for fetchDynamicValues
* Add table header index prop for columns selector
* migration partial commit
* migration partial commit
* Refactor: removed unused import
* Update: reused function for checking dynamic value
* Update: removed unused import
* Fix format JSON issues
* Retrieve binding paths from entity selector components
* Fixes 6 remaining issues with UQI implementation
* Fix dropdown issues
* Fix dropdown height issues and fixes triggering of APIs when option is deselected
* Migration changes
* Fix QA generated UQI issues
* Fix projection component height and route change logic
* Fix multi select dropdown placeholder text issue and json stringify issue with switching view types
* Reset entity type value when command value changes
* Test changes
* Review comments
* Moved migrations around
* Corrected import statement
* Added JSON schema migration
* Updated schema version
* perf improvements and filter dropdown options feature
* Fix Code mirror component config for toggleComponentToJson input fields.
* Fix prettier issues
* fix prettier issues
* Fix style issues as a result of the merged conflicts
* Fix failing test case
* Fixed a few other flows (#14225)
* Fixed a few other flows
* Review comments
* Fix generate CRUD, fix evaluation of dynamic bindings and fix various styling issues.
* More fixes (#14367)
* Factor in the root formconfig parent key.
* Fix flickering issues, and evaluatedFormConfig issues
* fix: Teeny bugs (#14455)
* Teeny bugs
* Added previous functionality as is
* Improvements in the way we fetch dynamic values
* Fix stringiification issue and cyclic dependency issues
* Resolve projection component values deletion
* Resolve merge conflicts and fix prettier issues
* fix: Tsc issues
* Fix property pane connection navigation
* updating ee locator
* updating inputfield locator
* dropdown locator update
* Merge conflict not properly resolved.
* Fix s3 spec
* Fix Mongo Spec
* Fix some more tests
* fix: prevent cyclic dependency when switching to js mode (#14668)
* add delete events for change from array to string in diff
* add test to assert absence of cyclic dependency error when switching to js in switchgroup widget
* Assert that evaluation is not disabled when no cyclic dependency happens
* Cypress test preparations for google sheets and form controls
* Fixed a few test errors (#14874)
* Add: unit tests for uqi UI updates
- view type tests
- conditional output extraction
- processing conditional output to handle view/enabled state of the component
* Add: completed isValidFormConfig test
* Update: improved tests for update config
- These tests cover the functionality to update a section config after it's components are done evaluating
* Fix failing cypress tests and cyclic dependency issue
* Fixes some more tests
* Fixed migration of row objects (#14896)
* Bumped the version of design system package
* Update: reverted change to EE selector
* Fix deletion pointer
* Update: selector for js on load spec
- Synced with changes related to ADS dropdown
* Fix mongoDBShoppingCart spec
* Remove comments
* Fix: mongo shopping cart test failures
* fix: mongo shopping cart spec
* Dummy push to retrigger vercel
* fix: mongo shopping cart spec
* Update MongoDBShoppingCart_spec.js
* fix: removed unused click away
* dummy commit
* Update: moved helper functions to separate file
* Add: added tests for saga functions
- Worked on testing for
- extractFetchDynamicValueFormConfigs
- extractQueueOfValuesToBeFetched
* Add if check for queueOfValuesToBeFetched
* Resolve review comments
* Empty-Commit
Co-authored-by: Irongade <adeoluayangade@yahoo.com>
Co-authored-by: Ayush Pahwa <ayush@appsmith.com>
Co-authored-by: Aman Agarwal <aman@appsmith.com>
Co-authored-by: Ayangade Adeoluwa <37867493+Irongade@users.noreply.github.com>
Co-authored-by: Aishwarya UR <aishwarya@appsmith.com>
Co-authored-by: Favour Ohanekwu <fohanekwu@gmail.com>
Co-authored-by: Albin <albin@appsmith.com>
* Show js function execution error logs
* remove unused function
* improve check for async functions
* clear errors for deleted jsActions
* fix typescript error
* modify js function execution error logging
* test that execution parse errors are logged in the debugger
* Add test to show that js execution errors are logged in the debugger
* re-order js execution tests
* Add type to jsObj variable
* update cypress tests
* update cypress test
- Change the parsing logic to track variableDeclarator and check for JSObject being declared and find its properties using the AST Node.
- As we are exactly checking JSObject Node we won’t have any incorrect properties defined or any property missed
* parsing of js object has changed
* added test for parsing
* parse object with literal
* added variable to reactive paths and solved some PR comments
* test fixed
* small fix
* removed restricting functions to be added if object body has errors
Co-authored-by: Aishwarya UR <aishwarya@appsmith.com>
