List view
Create simple example of OpenTelemetry project with postgres, express and redis.
Overdue by 1 year(s)•Due by August 13, 2024•1/1 issues closedIt may be useful to run some benchmarks to show the performance impact of using the Logger and OTel plugins. As the Otel plugin will require to perform some asynchronous calls, we expect a significant performance degradation. This will be ok for the users that require Otel anyway, but it is important to know the tradeoffs. Also, it is a way to make sure that it performs as good as it can considering the additional overhead of using Otel.
Overdue by 1 year(s)•Due by October 11, 2024•0/1 issues closedWe need a comprehensive documentation for the plugin, that explains how to use it in different scenarios as well as a quick start guide and a reference API, which I guess will not be very large as the plugin should be a matter of instantiating it and passing it in BullMQs constructor. In the documentation we would also like to show some screenshots of examples when using the Otel on some dashboard, so that users can more easily understand what they should expect out of it.
Overdue by 1 year(s)•Due by September 14, 2024•1/1 issues closedThe goal of this milestone is to develop the Otel (https://opentelemetry.io) plugin by implementing the telemetry interface designed in previous milestones. This will be done in a separate repository and will be published as a separate NPM package (bullmq-otel for example). Only users interested in this plugin would need to install it. The plugin must have a complete test suite that covers all its functionality, to prove it works correctly.
Overdue by 1 year(s)•Due by September 27, 2024•0/1 issues closedIn this milestone, the documentation of the interface should be documented. Every method in the interface and maybe some small guide on how to implement a typical observability plugin. Furthermore, we could use the “log” integration as an example, although this logger implementation will lack important telemetry concepts such as “context”, “spans” and so on.
Overdue by 1 year(s)•Due by August 30, 2024A simple but convenient implementation of the Telemetry interface is the Logger. This integration will be useful to validate the interface, but also it would be useful to enable a powerful logger. We should probably use the "pino" logging library (https://www.npmjs.com/package/pino) for this integration as it is the fastest one, and we want to keep the telemetry as lightweight as possible. This integration should be created in a new module, "bullmq-logger" for example, so that it can be used optionally for those users that want to do so.
Overdue by 1 year(s)•Due by September 14, 2024The telemetry interface should be a Typescript interface that should be part of the BullMQ package. In this milestone it is expected to have a complete (or almost complete as a future implementation may prove that the interface must be changed), and some pseudo code showing the feasibility of the interface for implementing observability add-ons. The challenge here is that the interface must be used internally in BullMQ, for example, lets say that we enhance BullMQ Queue constructor to accept a Telemetry plugin, something like this: ``` import { Queue } from "bullmq"; import { Otel } from "bullmq-otel" const queue = new Queue("test-otel", { telemetry: new Otel(…)}); ``` So when giving a Telemetry plugin the queue will now call the interface methods when doing different operations, for example when the user calls queue.add(…), we may now need to call some method of the plugin to register that call and so on, and similarly with the Worker class in BullMQ. Some references for inspiration are these third party implementations: https://github.com/appsignal/opentelemetry-instrumentation-bullmq and https://github.com/jenniferplusplus/opentelemetry-instrumentation-bullmq These are based on monkey patching BullMQ, which is what we want to precisely avoid by defining this new interface. Here some summary from ChatGPT: To develop a Telemetry interface for BullMQ with the ability to integrate with frameworks like OpenTelemetry, it's crucial to understand some key OpenTelemetry concepts: Spans, Traces, and Context Propagation. ### Key Concepts: 1. **Spans**: - A span represents a unit of work or operation. Each span includes a name, start and end timestamps, attributes, and optionally events and links to other spans. - Spans can be nested to represent parent-child relationships, forming a trace. A trace is a collection of spans that share the same `trace_id` [oai_citation:1,Traces | OpenTelemetry](https://opentelemetry.io/docs/concepts/signals/traces/) [oai_citation:2,OpenTelemetry Concepts | OpenTelemetry](https://opentelemetry.io/docs/concepts/). 2. **Traces**: - A trace tracks the progression of a single request as it moves through various components of a distributed system. It consists of multiple spans that are related to each other through parent-child relationships [oai_citation:3,Traces | OpenTelemetry](https://opentelemetry.io/docs/concepts/signals/traces/) [oai_citation:4,OpenTelemetry Concepts | OpenTelemetry](https://opentelemetry.io/docs/concepts/). 3. **Context Propagation**: - Context propagation is the mechanism by which trace and span information is passed between different parts of a system. This ensures that the spans created in different parts of the system are correlated correctly and assembled into a trace [oai_citation:5,Context propagation | OpenTelemetry](https://opentelemetry.io/docs/concepts/context-propagation/) [oai_citation:6,Context | OpenTelemetry](https://opentelemetry.io/docs/specs/otel/context/).
Overdue by 1 year(s)•Due by September 16, 2024•3/3 issues closed- No due date•5/5 issues closed