Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Test all-in-one bundles #120

Closed
romani opened this issue Jul 14, 2024 · 9 comments
Closed

Test all-in-one bundles #120

romani opened this issue Jul 14, 2024 · 9 comments

Comments

@romani
Copy link
Member

romani commented Jul 14, 2024

After initial implementation of new workflow is done by Github, generate report for

We need to test all bundles to make sure that report generation can be finished.

Expected problems: oom errors due to numerous violations or timeouts.

If any problem detected, we need to reduce amount of projects to let report to finish.

PR to checkstyle-gsoc/checkstyle#28

@romani
Copy link
Member Author

romani commented Jul 14, 2024

We have a lot of bundles, so triggering should be done from phone on the go, not a laptop time.
It will be good metric on how convenient to use it

@romani
Copy link
Member Author

romani commented Jul 20, 2024

@relentless-pursuit , please confirm that you can test all bundles in your organization.

While we waiting for merge in main repository, we can reuse your org and simply get all bundles be tested (just to make sure reports can successfully finish).

If yes, please start. I will join.
We can have single PR for one Check.

@romani
Copy link
Member Author

romani commented Jul 20, 2024

We should start from running all-in-one first.
If it pass it means all others will pass for sure.

@relentless-pursuit
Copy link
Collaborator

@relentless-pursuit , please confirm that you can test all bundles in your organization.

While we waiting for merge in main repository, we can reuse your org and simply get all bundles be tested (just to make sure reports can successfully finish).

If yes, please start. I will join. We can have single PR for one Check.

@romani i don't mind testing but wouldn't it be great if we could test it in main repo. because, we already have a mechanism of backwards compatability, not that the merge will hinder the existing regression testing. But, if we do testing and it can be a tough task to test all 180 checks, how can we evaluate better if not done on a main repo?

@romani
Copy link
Member Author

romani commented Jul 20, 2024

wouldn't it be great if we could test it in main repo.

Yes, but it would not be quicker.
Moreover, it will use GitHub quotes of execution, so we will make huge queue and we will slow down all others.
In your org, you have more freedom and isolation.
We will test not a workflow , but ability of reports to finish, no timeouts.

But, if we do testing and it can be a tough task to test all 180 checks

Not though, just a bit long, but we can start now, and until September trigger 180 executions for all-in-one.
I will help to trigger from phone, as you already did all (except for one issue ) so convenient that it takes less 1 minutes to trigger.

@romani romani changed the title Test all bundles Test all-in-one bundles Jul 20, 2024
@relentless-pursuit
Copy link
Collaborator

relentless-pursuit commented Jul 20, 2024

Test PRs for below Checks

  • AbbreviationAsWordInName
  • AbstractClassName
  • AnnotationLocation
  • AnnotationOnSameLine
  • AnnotationUseStyle
  • AnonInnerLength
  • ArrayTrailingComma
  • ArrayTypeStyle
  • AtclauseOrder
  • AvoidDoubleBraceInitialization
  • AvoidEscapedUnicodeCharacters
  • AvoidInlineConditionals
  • AvoidNestedBlocks
  • AvoidNoArgumentSuperConstructorCall
  • AvoidStarImport
  • AvoidStaticImport
  • BooleanExpressionComplexity
  • CatchParameterName
  • ClassDataAbstractionCoupling
  • ClassFanOutComplexity
  • ClassMemberImpliedModifier
  • ClassTypeParameterName
  • CommentsIndentation
  • ConstantName
  • ConstructorsDeclarationGrouping
  • CovariantEquals
  • CustomImportOrder
  • CyclomaticComplexity
  • DeclarationOrder
  • DefaultComesLast
  • DescendantToken defect in Check https://github.com/checkstyle-gsoc/checkstyle/actions/runs/11033049153/job/30643403949 but we need to avoid in list of projects problematic projects or sources
  • DesignForExtension
  • EmptyBlock
  • EmptyCatchBlock
  • EmptyForInitializerPad
  • EmptyForIteratorPad
  • EmptyLineSeparator
  • EmptyStatement
  • EqualsAvoidNull
  • EqualsHashCode
  • ExecutableStatementCount
  • ExplicitInitialization
  • FallThrough
  • FileLength
  • FileTabCharacter
  • FinalClass
  • FinalLocalVariable
  • FinalParameters
  • GenericWhitespace
  • Header failng https://github.com/checkstyle-gsoc/checkstyle/actions/runs/10871292828/job/30164960433
  • HiddenField
  • HideUtilityClassConstructor
  • IllegalCatch
  • IllegalIdentifierName
  • IllegalImport
  • IllegalInstantiation
  • IllegalThrows
  • IllegalToken
  • IllegalTokenText - failure https://github.com/checkstyle-gsoc/checkstyle/actions/runs/10862783821/job/30146211874
  • IllegalType
  • ImportOrder
  • Indentation
  • InnerAssignment
  • InnerTypeLast
  • InterfaceIsType
  • InterfaceMemberImpliedModifier - failure https://github.com/checkstyle-gsoc/checkstyle/actions/runs/10858509842
  • InterfaceTypeParameterName
  • InvalidJavadocPosition
  • JavaNCSS
  • JavadocBlockTagLocation
  • JavadocContentLocation
  • JavadocLeadingAsteriskAlign
  • JavadocMethod
  • JavadocMissingLeadingAsterisk
  • JavadocMissingWhitespaceAfterAsterisk
  • JavadocPackage
  • JavadocParagraph
  • JavadocStyle
  • JavadocTagContinuationIndentation
  • JavadocType
  • JavadocVariable
  • LambdaBodyLength
  • LambdaParameterName
  • LeftCurly
  • LineLength
  • LocalFinalVariableName
  • LocalVariableName
  • MagicNumber
  • MatchXpath
  • MemberName
  • MethodCount
  • MethodLength
  • MethodName
  • MethodParamPad
  • MethodTypeParameterName
  • MissingCtor
  • MissingDeprecated
  • MissingJavadocMethod
  • MissingJavadocPackage
  • MissingJavadocType
  • MissingOverride
  • MissingSwitchDefault
  • ModifiedControlVariable
  • ModifierOrder
  • MultipleStringLiterals - failure Official Testing of regression-report.yml checkstyle-gsoc/checkstyle#28 (comment)
  • MultipleVariableDeclarations
  • MutableException
  • NPathComplexity
  • NeedBraces Issue #15322: NeedBraces check should violate switch expressions with a… checkstyle#15334
  • NestedForDepth
  • NestedIfDepth
  • NestedTryDepth
  • NewlineAtEndOfFile
  • NoArrayTrailingComma
  • NoClone
  • NoCodeInFile
  • NoEnumTrailingComma
  • NoFinalizer
  • NoLineWrap
  • NoWhitespaceAfter
  • NoWhitespaceBefore
  • NoWhitespaceBeforeCaseDefaultColon
  • NonEmptyAtclauseDescription
  • OneStatementPerLine
  • OneTopLevelClass
  • OperatorWrap
  • OrderedProperties
  • OuterTypeFilename
  • OuterTypeNumber
  • OverloadMethodsDeclarationOrder
  • PackageAnnotation
  • PackageDeclaration
  • PackageName
  • ParameterAssignment
  • ParameterName
  • ParameterNumber
  • ParenPad
  • PatternVariableName
  • RecordComponentName
  • RecordComponentNumber
  • RecordTypeParameterName
  • RedundantImport
  • RedundantModifier
  • Regexp
  • RegexpHeader
  • RegexpMultiline
  • RegexpOnFilename
  • RegexpSingleline
  • RegexpSinglelineJava
  • RequireEmptyLineBeforeBlockTagGroup
  • RequireThis
  • ReturnCount
  • RightCurly
  • SeparatorWrap
  • SimplifyBooleanExpression
  • SimplifyBooleanReturn
  • SingleLineJavadoc
  • SingleSpaceSeparator
  • StaticVariableName
  • StringLiteralEquality
  • SummaryJavadoc
  • SuperClone
  • SuperFinalize
  • SuppressWarnings
  • ThrowsCount
  • TodoComment
  • TrailingComment timeouted https://github.com/checkstyle-gsoc/checkstyle/actions/runs/10105466649/job/27945999431
  • Translation
  • TypeName
  • TypecastParenPad
  • UncommentedMain
  • UniqueProperties
  • UnnecessaryParentheses
  • UnnecessarySemicolonAfterOuterTypeDeclaration
  • UnnecessarySemicolonAfterTypeMemberDeclaration
  • UnnecessarySemicolonInEnumeration
  • UnnecessarySemicolonInTryWithResources
  • UnusedImports
  • UnusedLocalVariable
  • UpperEll
  • VariableDeclarationUsageDistance
  • VisibilityModifier
  • WhitespaceAfter
  • WhitespaceAround - there is oom during execution Test all-in-one from bottom to top checkstyle-gsoc/checkstyle#17 (comment)
  • WriteTag

@relentless-pursuit
Copy link
Collaborator

is this required?

@romani
Copy link
Member Author

romani commented Sep 1, 2024

Yes, as I already noticed RegexpHeader is not working and few others

@romani
Copy link
Member Author

romani commented Sep 27, 2024

Testing is finished at checkstyle-gsoc/checkstyle#30 (comment)

All failed config will be handled at #141

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
No open projects
Development

No branches or pull requests

2 participants