fix: compress pipeline graphs before sending to mermaid #8767

lbux · 2025-01-24T06:11:55Z

Related Issues

fixes Mermaid Crashes If trying to draw a large pipeline #8089

Proposed Changes:

When pipelines get too big, the resulting base64 output is too long when we send a request to the mermaid server leading to a HTTP 400 error. Mermaid supports compressed URLS using Pako. With zlib, we are able to compress our input in the expected format and send it off to Mermaid.

How did you test it?

I wanted to do as close to a 1:1 change so that this has 0 impact to the user. The same tests were used which hopefully means the changes are 1:1 as they have all passed.

I am also asking for tests from Haystack users with big pipelines on Discord.

Notes for the reviewer

I will make another PR that will use this change as a foundation for:

Extension parameters (jpg, png, webp, svg, pdf)
Background color modifications
Themes
Image size

Aaaaand.... offline support! #7896

Checklist

I have read the contributors guidelines and the code of conduct
I have updated the related issue with new insights and changes
I added unit tests and updated the docstrings
I've used one of the conventional commit types for my PR title: fix:, feat:, build:, chore:, ci:, docs:, style:, refactor:, perf:, test: and added ! in case the PR includes breaking changes.
I documented my code
I ran pre-commit hooks and fixed any issue

coveralls · 2025-01-24T06:17:03Z

Pull Request Test Coverage Report for Build 13008953763

Details

0 of 0 changed or added relevant lines in 0 files are covered.
2 unchanged lines in 1 file lost coverage.
Overall coverage increased (+0.002%) to 91.352%

Files with Coverage Reduction	New Missed Lines	%
core/pipeline/draw.py	2	97.1%

Totals
Change from base Build 13007414121:	0.002%
Covered Lines:	8863
Relevant Lines:	9702

💛 - Coveralls

davidsbatista

LGTM - I just added small suggestion regarding the new variable name

haystack/core/pipeline/draw.py

davidsbatista · 2025-01-27T11:55:00Z

Do you have an example you can share of such a big pipeline? How many components, connections?

Co-authored-by: David S. Batista <[email protected]>

lbux · 2025-01-27T17:48:11Z

Do you have an example you can share of such a big pipeline? How many components, connections?

I have an example locally. I will get back to you once I have access to the machine with the example.

lbux · 2025-01-27T20:38:53Z

Do you have an example you can share of such a big pipeline? How many components, connections?

21 components. I'll provide a YML file, but it is quite old. I've made significant changes to it, but I am comfortable sharing this older one I was using when I first encountered the issue.

components:
  bm25_retriever:
    init_parameters:
      document_store:
        init_parameters:
          bm25_algorithm: BM25L
          bm25_parameters: &id001 {}
          bm25_tokenization_regex: (?u)\b\w\w+\b
          embedding_similarity_function: cosine
          index: d68effa3-2651-414f-8081-624a3b6be7b3
        type: haystack.document_stores.in_memory.document_store.InMemoryDocumentStore
      filter_policy: replace
      filters: null
      scale_score: false
      top_k: 10
    type: haystack.components.retrievers.in_memory.bm25_retriever.InMemoryBM25Retriever
  context_router:
    init_parameters:
      huggingface_pipeline_kwargs:
        device: cuda:0
        model: MoritzLaurer/deberta-v3-base-zeroshot-v1.1-all-33
        task: zero-shot-classification
      labels:
      - Conversation Context
      - Direct Query
      token:
        env_vars:
        - HF_API_TOKEN
        - HF_TOKEN
        strict: false
        type: env_var
    type: haystack.components.routers.zero_shot_text_router.TransformersZeroShotTextRouter
  document_joiner:
    init_parameters:
      join_mode: concatenate
      sort_by_score: true
      top_k: null
      weights: null
    type: haystack.components.joiners.document_joiner.DocumentJoiner
  embedding_retriever:
    init_parameters:
      document_store:
        init_parameters:
          bm25_algorithm: BM25L
          bm25_parameters: *id001
          bm25_tokenization_regex: (?u)\b\w\w+\b
          embedding_similarity_function: cosine
          index: d68effa3-2651-414f-8081-624a3b6be7b3
        type: haystack.document_stores.in_memory.document_store.InMemoryDocumentStore
      filter_policy: replace
      filters: null
      return_embedding: false
      scale_score: false
      top_k: 10
    type: haystack.components.retrievers.in_memory.embedding_retriever.InMemoryEmbeddingRetriever
  list_to_str_adapter:
    init_parameters:
      custom_filters: {}
      output_type: str
      template: '{{ replies[0] }}'
      unsafe: false
    type: haystack.components.converters.output_adapter.OutputAdapter
  llm:
    init_parameters:
      generation_kwargs: {}
      keep_alive: null
      model: llama3.2:3b
      streaming_callback: null
      structured_format: null
      timeout: 120
      url: http://localhost:11434
    type: haystack_integrations.components.generators.ollama.chat.chat_generator.OllamaChatGenerator
  memory_joiner:
    init_parameters:
      type_: typing.List[haystack.dataclasses.chat_message.ChatMessage]
    type: haystack.components.joiners.branch.BranchJoiner
  memory_retriever:
    init_parameters:
      last_k: 10
      message_store:
        init_parameters: {}
        type: haystack_experimental.chat_message_stores.in_memory.InMemoryChatMessageStore
    type: haystack_experimental.components.retrievers.chat_message_retriever.ChatMessageRetriever
  memory_writer:
    init_parameters:
      message_store:
        init_parameters: {}
        type: haystack_experimental.chat_message_stores.in_memory.InMemoryChatMessageStore
    type: haystack_experimental.components.writers.chat_message_writer.ChatMessageWriter
  prompt_general_no_retrieval:
    init_parameters:
      required_variables:
      - query
      template:
      - content: "\nAnswer the query using your own knowledge.\nQuestion: {{ query\
          \ }} \nAnswer:\n"
        meta: {}
        name: null
        role: user
      variables:
      - query
    type: haystack.components.builders.chat_prompt_builder.ChatPromptBuilder
  prompt_joiner:
    init_parameters:
      type_: typing.List[haystack.dataclasses.chat_message.ChatMessage]
    type: haystack.components.joiners.branch.BranchJoiner
  prompt_no_rewrite_no_context:
    init_parameters:
      required_variables:
      - query
      - documents
      template:
      - content: """
        meta: {}
        name: null
        role: user
      variables:
      - query
      - documents
    type: haystack.components.builders.chat_prompt_builder.ChatPromptBuilder
  prompt_rewrite_context:
    init_parameters:
      required_variables:
      - query
      - memories
      template:
      - content: ""
        meta: {}
        name: null
        role: user
      variables:
      - query
      - memories
    type: haystack.components.builders.chat_prompt_builder.ChatPromptBuilder
  prompt_rewrite_context_retrieve:
    init_parameters:
      required_variables:
      - query
      - documents
      - memories
      template:
      - content: ""
        meta: {}
        name: null
        role: user
      variables:
      - query
      - documents
      - memories
    type: haystack.components.builders.chat_prompt_builder.ChatPromptBuilder
  query_rephrase_llm:
    init_parameters:
      generation_kwargs: {}
      keep_alive: null
      model: llama3.2:3b
      raw: false
      streaming_callback: null
      system_prompt: null
      template: null
      timeout: 120
      url: http://localhost:11434
    type: haystack_integrations.components.generators.ollama.generator.OllamaGenerator
  ranker:
    init_parameters:
      calibration_factor: 1.0
      device: null
      document_prefix: ''
      embedding_separator: '

        '
      meta_fields_to_embed: []
      model: cross-encoder/ms-marco-MiniLM-L-6-v2
      model_kwargs:
        device_map: cuda:0
      query_prefix: ''
      scale_score: true
      score_threshold: null
      token:
        env_vars:
        - HF_API_TOKEN
        - HF_TOKEN
        strict: false
        type: env_var
      tokenizer_kwargs: {}
      top_k: 5
    type: haystack.components.rankers.transformers_similarity.TransformersSimilarityRanker
  retrieval_router_context:
    init_parameters:
      huggingface_pipeline_kwargs:
        device: cuda:0
        model: MoritzLaurer/deberta-v3-base-zeroshot-v1.1-all-33
        task: zero-shot-classification
      labels:
      - Retrieve Documents
      - Context Only
      token:
        env_vars:
        - HF_API_TOKEN
        - HF_TOKEN
        strict: false
        type: env_var
    type: haystack.components.routers.zero_shot_text_router.TransformersZeroShotTextRouter
  retrieval_router_general:
    init_parameters:
      huggingface_pipeline_kwargs:
        device: cuda:0
        model: MoritzLaurer/deberta-v3-base-zeroshot-v1.1-all-33
        task: zero-shot-classification
      labels:
      - Retrieve Documents
      - General Knowledge
      token:
        env_vars:
        - HF_API_TOKEN
        - HF_TOKEN
        strict: false
        type: env_var
    type: haystack.components.routers.zero_shot_text_router.TransformersZeroShotTextRouter
  rewrite_llm_prompt:
    init_parameters:
      required_variables:
      - query
      - memories
      template: ""
      variables:
      - query
      - memories
    type: haystack.components.builders.prompt_builder.PromptBuilder
  router_output_joiner:
    init_parameters:
      type_: str
    type: haystack.components.joiners.branch.BranchJoiner
  text_embedder:
    init_parameters:
      batch_size: 32
      config_kwargs: null
      device:
        device: cuda:0
        type: single
      model: Alibaba-NLP/gte-large-en-v1.5
      model_kwargs: null
      normalize_embeddings: false
      precision: float32
      prefix: ''
      progress_bar: true
      suffix: ''
      token:
        env_vars:
        - HF_API_TOKEN
        - HF_TOKEN
        strict: false
        type: env_var
      tokenizer_kwargs: null
      truncate_dim: null
      trust_remote_code: true
    type: haystack.components.embedders.sentence_transformers_text_embedder.SentenceTransformersTextEmbedder
connections:
- receiver: retrieval_router_context.text
  sender: context_router.Conversation Context
- receiver: retrieval_router_general.text
  sender: context_router.Direct Query
- receiver: router_output_joiner.value
  sender: retrieval_router_context.Retrieve Documents
- receiver: router_output_joiner.value
  sender: retrieval_router_context.Context Only
- receiver: prompt_no_rewrite_no_context.query
  sender: retrieval_router_general.Retrieve Documents
- receiver: prompt_general_no_retrieval.query
  sender: retrieval_router_general.General Knowledge
- receiver: rewrite_llm_prompt.memories
  sender: memory_retriever.messages
- receiver: prompt_rewrite_context.memories
  sender: memory_retriever.messages
- receiver: prompt_rewrite_context_retrieve.memories
  sender: memory_retriever.messages
- receiver: memory_writer.messages
  sender: memory_joiner.value
- receiver: ranker.documents
  sender: document_joiner.documents
- receiver: embedding_retriever.query_embedding
  sender: text_embedder.embedding
- receiver: document_joiner.documents
  sender: embedding_retriever.documents
- receiver: prompt_rewrite_context_retrieve.documents
  sender: ranker.documents
- receiver: prompt_no_rewrite_no_context.documents
  sender: ranker.documents
- receiver: list_to_str_adapter.replies
  sender: query_rephrase_llm.replies
- receiver: memory_joiner.value
  sender: llm.replies
- receiver: prompt_joiner.value
  sender: prompt_no_rewrite_no_context.prompt
- receiver: prompt_joiner.value
  sender: prompt_rewrite_context.prompt
- receiver: prompt_joiner.value
  sender: prompt_rewrite_context_retrieve.prompt
- receiver: prompt_joiner.value
  sender: prompt_general_no_retrieval.prompt
- receiver: query_rephrase_llm.prompt
  sender: rewrite_llm_prompt.prompt
- receiver: prompt_rewrite_context.query
  sender: list_to_str_adapter.output
- receiver: rewrite_llm_prompt.query
  sender: router_output_joiner.value
- receiver: llm.messages
  sender: prompt_joiner.value
max_runs_per_component: 100
metadata: {}

davidsbatista · 2025-01-28T11:03:17Z

I thought about adding another integration test with a big pipeline, but I think it's enough to see that now sending the graph compressed still works as expected.

vblagoje · 2025-01-28T12:08:54Z

Wow nice - cool stuff @lbux

lbux and others added 2 commits January 23, 2025 21:57

compress graph data to support pako endpoint

692dc48

Merge branch 'deepset-ai:main' into draw-offline

779d50b

lbux requested review from a team as code owners January 24, 2025 06:11

lbux requested review from dfokina and davidsbatista and removed request for a team January 24, 2025 06:11

github-actions bot added the topic:core label Jan 24, 2025

lbux changed the title ~~Draw offline~~ fix: compress pipeline graphs before sending to mermaid Jan 24, 2025

lbux mentioned this pull request Jan 27, 2025

feat: Add support for custom (or offline) Mermaid.ink server and support all parameters #8772

Open

davidsbatista approved these changes Jan 27, 2025

View reviewed changes

haystack/core/pipeline/draw.py Outdated Show resolved Hide resolved

haystack/core/pipeline/draw.py Outdated Show resolved Hide resolved

lbux and others added 2 commits January 27, 2025 09:47

Update haystack/core/pipeline/draw.py

7d1c9d7

Co-authored-by: David S. Batista <[email protected]>

Update haystack/core/pipeline/draw.py

4516ad8

Co-authored-by: David S. Batista <[email protected]>

Merge branch 'main' into draw-offline

a59729e

davidsbatista merged commit d939321 into deepset-ai:main Jan 28, 2025
18 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix: compress pipeline graphs before sending to mermaid #8767

fix: compress pipeline graphs before sending to mermaid #8767

lbux commented Jan 24, 2025 •

edited

Loading

coveralls commented Jan 24, 2025 •

edited

Loading

davidsbatista left a comment

davidsbatista commented Jan 27, 2025

lbux commented Jan 27, 2025

lbux commented Jan 27, 2025 •

edited

Loading

davidsbatista commented Jan 28, 2025

vblagoje commented Jan 28, 2025

fix: compress pipeline graphs before sending to mermaid #8767

fix: compress pipeline graphs before sending to mermaid #8767

Conversation

lbux commented Jan 24, 2025 • edited Loading

Related Issues

Proposed Changes:

How did you test it?

Notes for the reviewer

Checklist

coveralls commented Jan 24, 2025 • edited Loading

Pull Request Test Coverage Report for Build 13008953763

Details

💛 - Coveralls

davidsbatista left a comment

Choose a reason for hiding this comment

davidsbatista commented Jan 27, 2025

lbux commented Jan 27, 2025

lbux commented Jan 27, 2025 • edited Loading

davidsbatista commented Jan 28, 2025

vblagoje commented Jan 28, 2025

lbux commented Jan 24, 2025 •

edited

Loading

coveralls commented Jan 24, 2025 •

edited

Loading

lbux commented Jan 27, 2025 •

edited

Loading