fix: Fix server side skip test cases (#36572)

## Description
> [!TIP]  
> _Add a TL;DR when the description is longer than 500 words or
extremely technical (helps the content, marketing, and DevRel team)._
>
> _Please also include relevant motivation and context. List any
dependencies that are required for this change. Add links to Notion,
Figma or any other documents that might be relevant to the PR._


Fixes #`Issue Number`  
_or_  
Fixes `Issue URL`
> [!WARNING]  
> _If no issue exists, please create an issue first, and check with the
maintainers if the issue is valid._

## Automation

/ok-to-test tags="@tag.IDE"

### 🔍 Cypress test results
<!-- This is an auto-generated comment: Cypress test results  -->
> [!TIP]
> 🟢 🟢 🟢 All cypress tests have passed! 🎉 🎉 🎉
> Workflow run:
<https://github.com/appsmithorg/appsmith/actions/runs/11370534880>
> Commit: dea75ff1999643d072745a38e91f741677ac5703
> <a
href="https://internal.appsmith.com/app/cypress-dashboard/rundetails-65890b3c81d7400d08fa9ee5?branch=master&workflowId=11370534880&attempt=1"
target="_blank">Cypress dashboard</a>.
> Tags: `@tag.IDE`
> Spec:
> <hr>Wed, 16 Oct 2024 17:40:56 UTC
<!-- end of auto-generated comment: Cypress test results  -->


## Communication
Should the DevRel and Marketing teams inform users about this change?
- [ ] Yes
- [x] No


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

- **Bug Fixes**
- Improved handling and reporting of test failures during the build
process.
	- Enhanced logging for better clarity on test results.

- **New Features**
- Introduced detailed output for failed and skipped tests, now available
in the GitHub step summary and as comments on pull requests.
- Added new input parameter `is-pg-build` for enhanced workflow control.
- Updated default value for `skip-tests` to "false" to enforce test
execution.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
This commit is contained in:
Sagar Khalasi 2024-10-17 10:30:42 +05:30 committed by GitHub
parent 7ff703c7de
commit 81b2146cab
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194

View File

@ -183,35 +183,100 @@ jobs:
else
export APPSMITH_DB_URL="mongodb://localhost:27017/mobtools"
fi
args=()
if [[ "${{ steps.run_result.outputs.run_result }}" == "failedtest" ]]; then
failed_tests="${{ steps.failed_tests.outputs.tests }}"
args+=("-DfailIfNoTests=false" "-Dsurefire.failIfNoSpecifiedTests=false" "-Dtest=${failed_tests}")
fi
if ! mvn test "${args[@]}"; then
echo "Generating failed test list:"
failed_tests_file="$PWD/failed-server-tests.txt"
# Run tests and capture logs
mvn test "${args[@]}" | tee mvn_test.log
# Check for "BUILD FAILURE" in the mvn_test.log
if grep -q "BUILD FAILURE" mvn_test.log; then
test_result="failed"
else
test_result="passed"
fi
echo "test_result variable value: ${test_result}"
# Prepare output file for failed tests and ensure a fresh file is created
OUTPUT_FILE="failed-server-tests.txt"
rm -f "$OUTPUT_FILE"
touch "$OUTPUT_FILE"
failed_modules=()
skipped_modules=()
# Process mvn_test.log for FAILURE and SKIPPED statuses
while IFS= read -r line; do
if [[ $line == *"FAILURE"* ]]; then
module_name=$(echo "$line" | awk '{print $2}')
failed_modules+=("$module_name")
elif [[ $line == *"SKIPPED"* ]]; then
module_name=$(echo "$line" | awk '{print $2}')
skipped_modules+=("$module_name")
fi
done < mvn_test.log
echo "Failed Modules: ${failed_modules[*]}"
echo "Skipped Modules: ${skipped_modules[*]}"
# Handle older approach for reading failed tests from XML files
failed_tests_from_xml="$PWD/failed-tests-from-xml.txt"
gawk -F\" '/<testcase / {cur_test = $4 "#" $2} /<(failure|error) / {print cur_test}' $(find . -type f -name 'TEST-*.xml') \
| sort -u \
| tee "$failed_tests_file"
echo "Saved to $failed_tests_file"
if [[ -s $failed_tests_file ]]; then
| tee "$failed_tests_from_xml"
# Filter out failed modules and add only relevant tests to the final failed list
for module in "${failed_modules[@]}"; do
grep -v "$module" "$failed_tests_from_xml" > temp_file && mv temp_file "$failed_tests_from_xml"
done
# Include all skipped module test files in the final list
for module in "${skipped_modules[@]}"; do
module_directories=$(find . -path "*/${module}*/src/test/java/*" -type f -name "*Test.java" -exec dirname {} \; | sort -u)
for module_directory in $module_directories; do
test_classes=$(find "$module_directory" -type f -name "*Test.java" | sed 's|.*/src/test/java/||; s|\.java$||; s|/|.|g')
for class_name in $test_classes; do
if [[ ${#class_name} -le 240 ]] && ! grep -Fxq "$class_name#" "$OUTPUT_FILE"; then
echo "${class_name}#" >> "$OUTPUT_FILE"
fi
done
done
done
# Combine the XML file test cases and skipped module test files into the final output file
cat "$failed_tests_from_xml" >> "$OUTPUT_FILE"
# Print the final output
cat "$OUTPUT_FILE"
if [[ -s $OUTPUT_FILE ]]; then
content="$(
echo "## Failed server tests"
echo
sed 's/^/- /' "$failed_tests_file"
sed 's/^/- /' "$OUTPUT_FILE"
)"
echo "$content" >> "$GITHUB_STEP_SUMMARY"
# Add a comment to the PR with the list of failed tests.
# Post a comment to the PR
curl --silent --show-error \
--header "Authorization: Bearer ${{ secrets.GITHUB_TOKEN }}" \
--data "$(jq -n --arg body "$content" '$ARGS.named')" \
"https://api.github.com/repos/$GITHUB_REPOSITORY/issues/${{ inputs.pr }}/comments" \
> /dev/null
fi
# Fail the script if tests did not pass
if [[ "$test_result" == "failed" ]]; then
echo "Tests failed, exiting with status 1."
exit 1
fi
# Set status = failedtest
- name: Set fail if there are test failures
if: failure()