Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Staging to main #534

Merged
merged 191 commits into from
Jul 12, 2024
Merged

Staging to main #534

merged 191 commits into from
Jul 12, 2024

Conversation

aashipandya
Copy link
Collaborator

No description provided.

abhishekkumar-27 and others added 30 commits May 31, 2024 21:05
* Test IntegrationQA added

* update test cases

* update test

* update node count assertions

* test changes

* update changes

* modification test

* Code refatctor test cases

* Handle allowedlist issue in test

* test changes

* update test

* test case execution

* test chatbot updates

* test case update file

* added file

---------

Co-authored-by: Pravesh Kumar <[email protected]>
* Wikipedia source to accept all valid urls

* wikipedia url to support multiple languages

* integrated wiki langauge param for extract api

* Youtube video timestamps

---------

Co-authored-by: kartikpersistent <[email protected]>
* groq llm integration backend

* groq and description in node properties

* added groq in options

---------

Co-authored-by: kartikpersistent <[email protected]>
* expand

* minimize-icon

* css changes

* chat history

* chatbot wider Side Nav

* expand icon

* chatbot UI

* Delete

* merge fixes

* code suggestions

---------

Co-authored-by: kartikpersistent <[email protected]>
* chunks create before extraction using is_pre_process variable

* Return total pages for Model

* update requirement.txt

* total pages on uplaod API

* added the Confirmation Dialog

* added the selected files into the confirmation modal

* format and lint fixes

* added the stop watch image

* fileselection on alert dialog

* Add timeout in docker for gunicorn workers

* Add cancel icon to info popup (#384)

* Info Modal Changes

* css changes

* recent merges

* Integration_qa test (#375)

* Test IntegrationQA added

* update test cases

* update test

* update node count assertions

* test changes

* update changes

* modification test

* Code refatctor test cases

* Handle allowedlist issue in test

* test changes

* update test

* test case execution

* test chatbot updates

* test case update file

* added file

---------

Co-authored-by: Pravesh Kumar <[email protected]>

* fixed status blank issue

* Rendering the file name instead of link for gcs and s3 sources in the info modal

* added the default page size

* Convert is_cancelled value from string to bool

* Issue fixed Processed chunked as 0 when file re-process again

* Youtube timestamps (#386)

* Wikipedia source to accept all valid urls

* wikipedia url to support multiple languages

* integrated wiki langauge param for extract api

* Youtube video timestamps

---------

Co-authored-by: kartikpersistent <[email protected]>

* groq llm integration backend (#286)

* groq llm integration backend

* groq and description in node properties

* added groq in options

---------

Co-authored-by: kartikpersistent <[email protected]>

* Save Total Pages in DB

* Added total Pages

* file selection when we didn't select anything from Main table

* added the danger icon only for large files

* added the overflow for more files and file selection for all new files

* moved the interface to types

* added the icon accoroding to the source

* set total page for wiki and youtube

* h3 heading

* merge

* updated the alert on basis if total pages

* deleted chunks

* polling based on total pages

* isNan check

* large file based on file size for s3 and gcs

* file source in server side event

* time calculation based on chunks for gcs and s3

---------

Co-authored-by: kartikpersistent <[email protected]>
Co-authored-by: Prakriti Solankey <[email protected]>
Co-authored-by: abhishekkumar-27 <[email protected]>
Co-authored-by: aashipandya <[email protected]>
* crreate new endpoint populate_graph_schema and update the query for getting lables from DB

* Added main.py changes
* Added file name to the content  sent to LLM

* added chunk text in the response

* increased the docs parts sent to llm

* Modified graph query

* mardown rendering

* youtube starttime

* icons

* offset changes

* removed the files due to codespace space issue

---------

Co-authored-by: vasanthasaikalluri <[email protected]>
Co-authored-by: kartikpersistent <[email protected]>
… text given by user (#405)

* added the json

* added schema from text dialog

* integrated the schemaAPI

* added the alert

* resize fixes
* Fixed env/docker-compose for local deployments + README doc

* wrong place for ENV in README

* by default, removed langsmith + fixed knn score string to float

* by default, removed langsmith + fixed knn score string to float

* Fixed strings in docker-compose env

* Added requirements (neo4j 5.15 or later, APOC, and instructions for Neo4j Desktop)

* Missed the TIME_PER_PAGE env, was causing NaN issue in the approx time processing notification. fixed that
* all unstructured files

* responsiveness

* added file type

* added the extensions

* spell mistake

* ppt file changes

---------

Co-authored-by: kartikpersistent <[email protected]>
… text given by user with checkbox (#415)

* added the json

* added schema from text dialog

* integrated the schemaAPI

* added the alert

* resize fixes

* Extract schema using direct ChatOpenAI API and Chain

* integrated the checkbox for schema to text dialog

* Update SettingModal.tsx

---------

Co-authored-by: Pravesh Kumar <[email protected]>
kartikpersistent and others added 29 commits July 5, 2024 09:44
* removed version of pydantic

* updating embedding of entities in post processing

* schema extraction get llm correction

* added the create_entity_embedding task in post processing

---------

Co-authored-by: kartikpersistent <[email protected]>
* Chat mode entity show

* Legend changes

* chat-mode-changes

* schema changes
* LLM Model Config fixes

* Added ENTITY_EMBEDDING to docker compose yml

* qa and embedding config and local network for ollama

* updated readme and frontend constants for llms

---------

Co-authored-by: aashipandya <[email protected]>
* Added chat document filter

* sending seleected filenames for chat response

---------

Co-authored-by: vasanthasaikalluri <[email protected]>
* added the chat mode menu

* fixed closing of the menu on dialog
* Updated transformers and sentence-transformers version

* Updated more dependencies

* added versions to all dependencies, updated a lot of them

* added psutil

---------

Co-authored-by: aashipandya <[email protected]>
* Modified document select and type checking for models

* added tokens for llama3

* added manual token cutoff
* added graph mode

* added doc names parameter

* added the graph mode

* chat modes configurable and added the tabs for graph only mode

* changed the example env

* cypher cleaning

* fixed the default tab mode for graph mode

---------

Co-authored-by: vasanthasaikalluri <[email protected]>
* delete popup

* table changes

* Format fixes and added the warning status indicator for the 0 selected labels and rels

* menu changes

* entity dividers

* added the status indicator

---------

Co-authored-by: kartikpersistent <[email protected]>
* UX changes

* added the count for orphan node deletion popup

* increased time out for banner

* changed alignment of the buttons for web sources

* removed the orphan node check

* state fix for orphan node deletion

* Failed file GCS Issue fixed

* Added logging time, Comment gzipMiddleware and send total unconnected Nodes in exitsing API

* added the total orphan nodes count

* added the new icons

* Single graph request (#497)

* Modified query to docchunkentities

* all nodes

* UI changes

* Dropdown Changes from checkbox

* state management of nodes and relations

* common dropdown

---------

Co-authored-by: vasanthasaikalluri <[email protected]>

* LLMs from multiple providers (#496)

* added the new models

* addition of new llms

* added the bedrock model

* removed the extra

* llm name change

* added bedrock

* ollama added

* upgraded the dependencies

* notebook for different llms

* formatted llm.py

* passing model name in generate_graph_doc for gemini

* changed the example env

* Added model name for chat response to return

---------

Co-authored-by: kartikpersistent <[email protected]>
Co-authored-by: vasanthasaikalluri <[email protected]>

* removed version of pydantic (#499)

* LLMs for chat (#500)

* Added token usage

* modified token usage

* theme changes

* Refresh handle (#502)

* loading Graph

* refresh check

* show graph check

* lint fix

* UI for graph enhancement operations (#501)

* added the new image for web sources header

* added the GraphEnhancement Dialog

* added the entity extraction and orphan node deletion in tabs

* added the orphan nodes deletion table

* image size changes

* added the status indicator for graph schema settings

* added the description for the Entity extraction and orphan node deletion

* added the size for the orphan node deletion columns

* Resetting the state on unmount phase

---------

Co-authored-by: Prakriti Solankey <[email protected]>

* Lint fix

* chatmode changes

* Revert "chatmode changes"

This reverts commit 60e32c3.

* removed the gap between header and the tab content

* filestructure change

* node and document colum changes

* latest package upgradation

* Entity embed (#505)

* removed version of pydantic

* updating embedding of entities in post processing

* schema extraction get llm correction

* added the create_entity_embedding task in post processing

---------

Co-authored-by: kartikpersistent <[email protected]>

* Chat mode entity show (#504)

* Chat mode entity show

* Legend changes

* chat-mode-changes

* schema changes

* dropdown reset value

* refetching the orphan nodes after every deletion

* made the loading false in API failure

* LLM Model Config fixes (#508)

* LLM Model Config fixes

* Added ENTITY_EMBEDDING to docker compose yml

* qa and embedding config and local network for ollama

* updated readme and frontend constants for llms

---------

Co-authored-by: aashipandya <[email protected]>

* Chat Document filter (#514)

* Added chat document filter

* sending seleected filenames for chat response

---------

Co-authored-by: vasanthasaikalluri <[email protected]>

* Change chat mode view (#515)

* added the chat mode menu

* fixed closing of the menu on dialog

* added the tooltip on sidenav chat mode button

* fixed fileselection issue

* graph query fixes

* Updated transformers and sentence-transformers version (#507)

* Updated transformers and sentence-transformers version

* Updated more dependencies

* added versions to all dependencies, updated a lot of them

* added psutil

---------

Co-authored-by: aashipandya <[email protected]>

* Modified document select and type checking for models (#518)

* Modified document select and type checking for models

* added tokens for llama3

* added manual token cutoff

* Graph chat (#525)

* added graph mode

* added doc names parameter

* added the graph mode

* chat modes configurable and added the tabs for graph only mode

* changed the example env

* cypher cleaning

* fixed the default tab mode for graph mode

---------

Co-authored-by: vasanthasaikalluri <[email protected]>

* added the ID

* UI bug fixes (#527)

* delete popup

* table changes

* Format fixes and added the warning status indicator for the 0 selected labels and rels

* menu changes

* entity dividers

* added the status indicator

---------

Co-authored-by: kartikpersistent <[email protected]>

* lint issues

* env variable

---------

Co-authored-by: kartikpersistent <[email protected]>
Co-authored-by: Pravesh Kumar <[email protected]>
Co-authored-by: vasanthasaikalluri <[email protected]>
Co-authored-by: aashipandya <[email protected]>
Co-authored-by: Michael Hunger <[email protected]>
* format fixes and graph schema indication fix

* Update README.md

* added chat modes variable in env updated the readme

* spell fix

* added the chat mode in env table

* added the logos

* fixed the overflow issues

* removed the extra fix

* Fixed specific scenario  "when the text from schema closes it should reopen the previous modal"

* readme changes

* removed dev console logs

* added new retrieval query (#533)

* format fixes and tab rendering fix

* fixed the setting modal reopen issue

---------

Co-authored-by: kartikpersistent <[email protected]>
Co-authored-by: vasanthasaikalluri <[email protected]>
@aashipandya aashipandya merged commit ccf8f60 into main Jul 12, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

10 participants