Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add option to ignore cache #23

Merged
merged 2 commits into from
Oct 22, 2024
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 15 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,21 @@ reset: () => void;
perform: (steps: string | string[]) => Promise<any | any[]>;
```

### Additional Note

In addition to the operations history, Copilot maintains a repository-level cache. If you need to ignore the current cache for any reason (e.g., when adding an action to the testing framework driver), you can set the environment variable `COPILOT_OVERRIDE_CACHE` to "true" before running your tests. This will ensure that the current cache is not taken into consideration and will override the existing one.

```shell
export COPILOT_OVERRIDE_CACHE=true
```

If you want to disable the override after setting it to "true" and revert to using the cache, you can set `COPILOT_OVERRIDE_CACHE` to "false"

```shell
export COPILOT_OVERRIDE_CACHE=false
```


## Integration with Testing Frameworks

Detox Copilot requires two main components to work:
Expand Down
22 changes: 22 additions & 0 deletions src/actions/StepPerformer.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -73,6 +73,7 @@ describe('StepPerformer', () => {
promptResult?: string;
codeEvaluationResult?: any;
cacheExists?: boolean;
overrideCache?: boolean;
}

const setupMocks = ({
Expand All @@ -82,6 +83,7 @@ describe('StepPerformer', () => {
promptResult = 'generated code',
codeEvaluationResult = 'success',
cacheExists = false,
overrideCache = false,
}: SetupMockOptions = {}) => {
mockPromptHandler.isSnapshotImageSupported.mockReturnValue(isSnapshotSupported);
mockSnapshotManager.captureSnapshotImage.mockResolvedValue(
Expand All @@ -92,6 +94,10 @@ describe('StepPerformer', () => {
mockPromptHandler.runPrompt.mockResolvedValue(promptResult);
mockCodeEvaluator.evaluate.mockResolvedValue(codeEvaluationResult);

if (overrideCache) {
process.env.COPILOT_OVERRIDE_CACHE = "true";
}

const viewHierarchyHash = 'hash';
(crypto.createHash as jest.Mock).mockReturnValue({
update: jest.fn().mockReturnValue({
Expand Down Expand Up @@ -244,4 +250,20 @@ describe('StepPerformer', () => {
expect(mockCodeEvaluator.evaluate).not.toHaveBeenCalled();
expect(fs.writeFileSync).not.toHaveBeenCalled();
});

it('should not use cached prompt result if COPILOT_OVERRIDE_CACHE is enabled', async () => {
const intent = 'tap button';
setupMocks({ cacheExists: true, overrideCache: true });

const result = await stepPerformer.perform(intent);

expect(result).toBe('success');
// Should call runPrompt or createPrompt. Shouldn't use current cache, but override it
expect(mockPromptCreator.createPrompt).toHaveBeenCalled();
expect(mockPromptHandler.runPrompt).toHaveBeenCalled();
expect(mockCodeEvaluator.evaluate).toHaveBeenCalledWith('generated code', mockContext);
expect(fs.writeFileSync).toHaveBeenCalled(); // Need to save cache


LironMShemen marked this conversation as resolved.
Show resolved Hide resolved
});
});
7 changes: 6 additions & 1 deletion src/actions/StepPerformer.ts
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ import * as path from 'path';
import * as crypto from 'crypto';
import {extractCodeBlock} from '@/utils/extractCodeBlock';


LironMShemen marked this conversation as resolved.
Show resolved Hide resolved
export class StepPerformer {
private cache: Map<string, any> = new Map();
private readonly cacheFilePath: string;
Expand Down Expand Up @@ -62,6 +63,10 @@ export class StepPerformer {
return { snapshot, viewHierarchy, isSnapshotImageAttached };
}

private shouldOverrideCache() {
return process.env.COPILOT_OVERRIDE_CACHE === "true" || process.env.COPILOT_OVERRIDE_CACHE === "1";
}

private async generateCode(
step: string,
previous: PreviousStep[],
Expand All @@ -71,7 +76,7 @@ export class StepPerformer {
): Promise<string> {
const cacheKey = this.generateCacheKey(step, previous, viewHierarchy);

if (this.cache.has(cacheKey)) {
if (!this.shouldOverrideCache() && this.cache.has(cacheKey)) {
return this.cache.get(cacheKey);
} else {
const prompt = this.promptCreator.createPrompt(step, viewHierarchy, isSnapshotImageAttached, previous);
Expand Down
Loading