SEEDConsistencyChecker

Getting Started

Play with a certain method? (CMIF for example)

1. Clone the repository:

   git clone https://github.com/SEEDGuard/SEEDConsistencyChecker.git
   

2. Install the required dependencies:

   pip install -r requirements.txt
   

3. Download the pre-trained weights from this location:

   https://drive.google.com/file/d/124tuB-Txg1nfv6mp9oc8ZRg7UhzSjC_M/view?usp=sharing
   

4. Replace the pretrained_model_path variable with the pre-trained weights location in the below file:

   SEEDConsistencyChecker/core/CMIF/Utils/seed_checker.py
   

5. Run the file from SEEDConsistencyChecker directory:

   cd SEEDConsistencyChecker
   python main.py --method CMIF --input_dir 'path/to/input/folder/' --output_dir 'path/to/output/folder/'
   

   Output is stored in the path/to/output/folder/ directory.

To run the Testing script

Steps 1 and 2 will be the same.

3. Configure the input and output path in testing.py:

   input_dir = 'path/to/input/folder/'
   output_dir = 'path/to/output/folder/'
   

4. Run the testing script from SEEDConsistencyChecker directory:

   cd SEEDConsistencyChecker
   python test/CMIF/testing.py
   

Using Docker

1. Clone the repository:

   git clone https://github.com/SEEDGuard/SEEDConsistencyChecker.git
   

2. Make sure you have Docker installed and started, follow if not Install Docker.

3. Replace the pretrained_model_path variable with the pre-trained weights location in the below file:

   SEEDConsistencyChecker/core/CMIF/Utils/seed_checker.py
   pretrained_model_path = "/app/core/CMIF/utils/t5_classification_final_ep2.mdl"
   

4. Navigate to the folder consisting of Dockerfile

5. Build Docker Image

   docker build --tag your-image-name  .
   

6. Run the Docker image inside container
   Update your-image-name, your_method, dataset according to your configuration. For more info on available methods refer to our Methods list provided in README

   -docker run -it your-image-name --method your_method --input_dir dataset/input/ --output_dir dataset/output/
   

   Example:

   docker build -t cmif:1.1 .
   
   docker run -it cmif:1.1
   

7. Application is up and running. Output is generated and you can see them in the test folder

8. You can see the logs, files, status for the container on the Docker Desktop.

Contributing

SEEDConsistencyChecker thrives on community contributions. Whether you're interested in enhancing its security features, expanding the API, or improving the current functionality, your contributions are welcome. Please refer to our contribution guideline at CONTRIBUTING.md for more information on how to contribute. Also refer to our Docker template if you are coming up with new Methods for the task.

Paper List

Year-Id	Title	Venue Name	Replication Package	If Integrated	If confirmed with the orignal author?	Current Contributors
2024-2	Code Comment Inconsistency Detection Based on Confidence Learning	TSE	Link			mrhuggins03, David Wang
2024-1	How are We Detecting Inconsistent Method Names? An Empirical Study from Code Review Perspective	arXiv	Link
2023-5	SmartCoCo: Checking Comment-Code Inconsistency in Smart Contracts via Constraint Propagation and Binding.	ASE
2023-4	Inconsistent Defect Labels: Essence, Causes, and Influence.	TSE	Link
2023-3	Keeping Mutation Test Suites Consistent and Relevant with Long-Standing Mutants.	FSE
2023-2	Data Quality Matters: A Case Study of Obsolete Comment Detection	ICSE	Link			Kriti Patnala
2023-1	When to Say What: Learning to Find Condition-Message Inconsistencies	ICSE	Link	✔️		EZ7051
2022-1	Verification of Consistency Between Process Models, Object Life Cycles, and Context-Dependent Semantic Specifications.	TSE
2021-2	A Context-based Automated Approach for Method Name Consistency Checking and Suggestion.	ICSE	Link
2021-1	Deep Just-In-Time Inconsistency Detection Between Comments and Source Code	AAAI	Link	✔️		vigneskv
2020-1	A Fine-Grained Analysis on the Inconsistent Changes in Code Clones.	ICSME
2019-1	A Large-Scale Empirical Study on Code-Comment Inconsistencies	ICPC	Link