Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add PDF syntax to Rouge #2058

Open
wants to merge 9 commits into
base: master
Choose a base branch
from

Conversation

petervwyatt
Copy link

Please accept this lexer for PDF syntax (a.k.a. "COS syntax").

PDF (Portable Document Format) is an object-based declarative page description language that, in reality, is a random access, binary (non-text) format. It is formally defined by ISO 32000-2:2020 and corrected by errata (please do not refer to outdated legacy Adobe documentation!). However, with care text-centric PDFs (full or portions) can be created such as might be used in documentation. This token-based, forward lexing lexer is not intended to be used with binary real-world PDFs as that is not how real PDFs need to be lexed (this will also likely generate Ruby UTF-8 errors anyway!).

We wish to leverage this Rouge PDF parser upstream in current and future PDF ISO standards and specifications based on AsciiDoc via Metanorma for use with the many code fragment examples in the documentation.

@petervwyatt
Copy link
Author

BTW the PDFs added are fully functional PDFs that will work in products such as Adobe Acrobat Reader. Likely just need to rename with a .pdf extension. Also be careful with EOL conversions out of Github as PDFs are binary files! Normally I'd control this using .gitattributes file with *.pdf binary but because Rouge doesn't use file extensions this isn't possible.

@petervwyatt
Copy link
Author

Failure of linelint is against the 2 different sample functional PDF files in lib/rouge/demos/pdf and spec/visual/samples/pdf. This is because PDFs are not required to have an EOL on their last line (after the %%EOF) and the Rouge grammar must support this, which is why the samples are the way they are. If this is critical to fix then the EOL can be added but there will then be no test to ensure the grammar successfully processes PDFs without the EOL.

Added EOL to last line of PDF to pass linelint CI check used by Rouge. This is not required by real PDF files.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant