Skip to content

Commit

Permalink
Merge pull request #268 from aai-institute/267-docs-fixes
Browse files Browse the repository at this point in the history
Docs: Fix code highlighting in third-party libraries integration guide
  • Loading branch information
janwillemkl authored Feb 28, 2024
2 parents bad9dd8 + 4d07d8a commit 4869a3c
Show file tree
Hide file tree
Showing 3 changed files with 17 additions and 16 deletions.
22 changes: 12 additions & 10 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
default_language_version:
python: python3.11
repos:
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v4.5.0
Expand All @@ -13,27 +15,27 @@ repos:
rev: v1.8.0
hooks:
# See https://github.com/pre-commit/mirrors-mypy/blob/main/.pre-commit-hooks.yaml
- id: mypy
types_or: [ python, pyi ]
args: [--ignore-missing-imports, --scripts-are-modules]
- id: mypy
types_or: [python, pyi]
args: [--ignore-missing-imports, --scripts-are-modules]
- repo: https://github.com/astral-sh/ruff-pre-commit
rev: v0.2.0
rev: v0.2.2
hooks:
- id: ruff
args: [ --fix, --exit-non-zero-on-fix ]
args: [--fix, --exit-non-zero-on-fix]
- id: ruff-format
- repo: https://github.com/PyCQA/bandit
rev: 1.7.7
hooks:
- id: bandit
args: [-c, pyproject.toml]
additional_dependencies: ["bandit[toml]"]
- id: bandit
args: [-c, pyproject.toml]
additional_dependencies: ["bandit[toml]"]
- repo: https://github.com/jsh9/pydoclint
rev: 0.3.9
rev: 0.4.1
hooks:
- id: pydoclint
- repo: https://github.com/jazzband/pip-tools
rev: 7.3.0
rev: 7.4.0
hooks:
- id: pip-compile
name: pip-compile requirements-dev.txt
Expand Down
2 changes: 1 addition & 1 deletion docs/_code/polars_example.py
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,6 @@
us_lakes = lakes.filter(pl.col("Country") == "United States of America")

with fs.open(f"lakefs://quickstart/{tx.branch.id}/us_lakes.csv", "wb") as f:
us_lakes.write_csv(f)
us_lakes.write_csv(f) # (1)!

tx.commit(message="Add US lakes")
9 changes: 4 additions & 5 deletions docs/guides/integrations.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ See the Pandas documentation on [reading/writing remote files](https://pandas.py

The following code snippet illustrates how to read and write Pandas data frames in various formats from/to a lakeFS repository in the context of a [transaction](transactions.md):

```python hl_lines="10 12"
```python hl_lines="8 10"
--8<-- "docs/_code/pandas_example.py"
```

Expand All @@ -32,7 +32,7 @@ This allows DuckDB to transparently query and store data located in lakeFS repos

Similar to the example above, the following code snippet illustrates how to read and write data from/to a lakeFS repository in the context of a [transaction](transactions.md) through the [DuckDB Python API](https://duckdb.org/docs/api/python/overview.html){: target="_blank" rel="noopener"}:

```python hl_lines="6 11 13"
```python hl_lines="6 9 11"
--8<-- "docs/_code/duckdb_example.py"
```

Expand All @@ -48,8 +48,7 @@ The Python API wrapper for the Rust-based [Polars](https://pola-rs.github.io/pol

Again, the following code example demonstrates how to read a Parquet file and save a modified version back in CSV format to a lakeFS repository from Polars in the context of a [transaction](transactions.md):


```python hl_lines="10 13-14"
```python hl_lines="8 11-12"
--8<-- "docs/_code/polars_example.py"
```

Expand All @@ -63,6 +62,6 @@ PyArrow `read_*` and `write_*` functions take an explicit `filesystem` parameter

The following example code illustrates the use of lakeFS-spec with PyArrow, reading a Parquet file and writing it back to a lakeFS repository as a partitioned CSV dataset in the context of a [transaction](transactions.md):

```python hl_lines="12 17"
```python hl_lines="10 15"
--8<-- "docs/_code/pyarrow_example.py"
```

0 comments on commit 4869a3c

Please sign in to comment.