As software interns these past few summers, we have continually encountered challenges learning the tools, procedures, and commands specific to the company at which we were working. Navigating proprietary commands and facing undocumented bugs have always made for a steep learning curve. The company wiki that is available is often disorganized, scattered, and sometimes deprecated. A more organized and updated company wiki system would not only accelerate our onboarding process but also allow employees alike to work more efficiently and autonomously.
To combat these hurdles, we set out to develop DevMate, an AI-powered assistant that integrates a finely-tuned chatbot with project board management. The chatbot is uniquely trained on a comprehensive set of resources: from wikis and code repositories to academic papers and project board issues. It creates embeddings, and uses caching and memory to store conversation history and optimize our requests. Our current proof of concept is specific to the IMAGE Project at McGill University’s Shared Reality Lab. This chatbot assistant can be applied to any organization’s documentation to provide their developers with a helpful aid in learning and navigating company software. It efficiently answers questions, provides direction, and offers interactive troubleshooting and debugging assistance.
Following a conversation with DevMate, if a user reports that they have solved their issue, DevMate parses through the company’s project board to see if other people have reported similar problems and, if so, it posts an automated comment summarizing the solution that was reached. If a user is not able to solve their problem with DevMate, they have the option to create a new issue on the project board automatically.
This project naturally integrated many different softwares and tools for it to come fully together. One of the largest parts of our project was figuring out how to train a LLM with the unique data of IMAGE. We experimented with several different LLMs and corresponding APIs for our chatbot, including GPT, DaVinci, Google, and Llama. We ran into many issues with the limitations of our computers and the constraints of available free APIs. While most of the models were very slow or had limitations on requests per minute, we ended up choosing GPT 3.5 for its superior performance.