Add thread-safe JSON storage layer for comments and users

[Copilot]

The repository had ad-hoc, non-thread-safe file I/O for comments and user data scattered throughout app.py, which also contained duplicate Flask app initialization.

Changes

New data_store.py module

• Thread-safe JSON persistence with threading.Lock for comments and users
• Clean API: load_comments(), add_comment(), load_users(), add_user(), authenticate_user(), etc.
• Graceful handling of corrupted/missing JSON files

Refactored app.py

• Removed duplicate Flask app initialization and inline load/save functions
• Integrated data_store module for all comment and user operations

Tests

• 21 tests for data_store (CRUD, thread safety, data integrity)
• 12 tests for Flask routes (auth, comments, bills)

Cleanup

• Added .gitignore (excludes .venv, node_modules, __pycache__)
• Fixed requirements.txt format (had duplicate/malformed entries)

Usage

import data_store

# Users
data_store.add_user('alice', 'password123', is_admin=False)
user = data_store.authenticate_user('alice', 'password123')

# Comments
data_store.add_comment('B001', 'Great bill!', 'alice')
comments = data_store.get_comments('B001')

Original prompt

Goal: Clean up the repository and implement a reliable JSON-based storage backend for (1) comments on bills and (2) user login information (user accounts and authentication), replacing any ad-hoc or file formats currently used. Create a single, well-tested, thread-safe, easy-to-use data layer and integrate it into the existing application routes and modules.

Tasks (detailed and actionable):

1. Repo audit and cleanup

• Scan the entire repository to find all places where comments on bills and user login/authentication are currently stored, loaded, or modified. This includes any files, modules, or Flask/Django routes, scripts, or utilities.
• Identify and remove or refactor duplicate or unused code related to comment and user data storage.
• Fix obvious lint/formatting issues in modified files (apply black/flake8 style fixes where applicable).

2. Add JSON storage layer

• Add a new package module at src/data_store.py (or project root if no src) that provides a clear API:
  • load_comments() -> dict: loads a comments mapping from data/comments.json (example format: {"<bill_id>":[{"id":...,"author":...,"text":...,"created_at":...}, ...], ...})
  • save_comments(comments: dict) -> None: atomically writes comments to data/comments.json
  • add_comment(bill_id: str, author: str, text: str) -> dict: creates a new comment object with uuid, timestamp, stores it for bill_id, persists, and returns the comment
  • get_comments_for_bill(bill_id: str) -> list
  • load_users() -> dict: loads users from data/users.json (format: {"username": {"password_hash":..., "created_at":..., "display_name":...}, ...})
  • save_users(users: dict) -> None
  • create_user(username: str, password: str, display_name: Optional[str]) -> dict: create a user with secure password hashing (bcrypt or passlib), persist, return user record
  • authenticate_user(username: str, password: str) -> bool or user
  • update_password(username, new_password)
• Use secure password hashing (bcrypt via passlib or bcrypt). If adding a dependency, update requirements.txt.
• Use atomic file writes and simple file locking to prevent race conditions (use tempfile + os.replace for atomic write and optionally portalocker for locking). Keep dependencies minimal; implement a fallback locking mechanism if portalocker isn't available.
• Create the data directory data/ with initial files data/comments.json ({}), data/users.json ({}) and ensure they're created automatically if missing.

3. Integrate with application

• Replace current ad-hoc storage calls for comment creation, retrieval, and user register/login with the new data_store API. Update routes/controllers so that login and comment submission work against JSON storage.
• If the project uses sessions, preserve existing login/session behavior and only switch backend storage for user credentials.

4. Tests and examples

• Add unit tests for data_store functions (tests/test_data_store.py) covering creating users, authenticating, adding comments, retrieving comments, concurrent writes (if feasible).

5. Documentation

• Update README.md with instructions on the new JSON storage: location of data files, how to initialize, default file formats, and how to run tests.

6. Misc cleanup

• Remove unused files/temporary artifacts discovered during the audit.
• Add .gitignore entries for data/ if users should not commit their local data files (but leave example files like data/comments.json.example committed). Add data/comments.json.example and data/users.json.example if appropriate.

Constraints and quality requirements:

• Do not change application behavior except for storage backend and code cleanups.
• Use secure password hashing; do not store plaintext passwords.
• Ensure atomic writes to JSON files to avoid corruption with concurrent requests.
• Keep changes minimal and well-documented; add comments where important.

Deliverable: Create a pull request that implements the above: adds the data_store module, creates data/ with example JSON files, updates routes to use the new API, adds unit tests, updates README, and performs small formatting/lint fixes. The PR title should be the problem title above and description should summarize the tasks completed and any migration steps for existing installations.

This pull request was created as a result of the following prompt from Copilot chat.

Goal: Clean up the repository and implement a reliable JSON-based storage backend for (1) comments on bills and (2) user login information (user accounts and authentication), replacing any ad-hoc or file formats currently used. Create a single, well-tested, thread-safe, easy-to-use data layer and integrate it into the existing application routes and modules.

Tasks (detailed and actionable):

1. Repo audit and cleanup

• Scan the entire repository to find all places where comments on bills and user login/authentication are currently stored, loaded, or modified. This includes any files, modules, or Flask/Django routes, scripts, or utilities.
• Identify and remove or refactor duplicate or unused code related to comment and user data storage.
• Fix obvious lint/formatting issues in modified files (apply black/flake8 style fixes where applicable).

2. Add JSON storage layer

• Add a new package module at src/data_store.py (or project root if no src) that provides a clear API:
  • load_comments() -> dict: loads a comments mapping from data/comments.json (example format: {"<bill_id>":[{"id":...,"author":...,"text":...,"created_at":...}, ...], ...})
  • save_comments(comments: dict) -> None: atomically writes comments to data/comments.json
  • add_comment(bill_id: str, author: str, text: str) -> dict: creates a new comment object with uuid, timestamp, stores it for bill_id, persists, and returns the comment
  • get_comments_for_bill(bill_id: str) -> list
  • load_users() -> dict: loads users from data/users.json (format: {"username": {"password_hash":..., "created_at":..., "display_name":...}, ...})
  • save_users(users: dict) -> None
  • create_user(username: str, password: str, display_name: Optional[str]) -> dict: create a user with secure password hashing (bcrypt or passlib), persist, return user record
  • authenticate_user(username: str, password: str) -> bool or user
  • update_password(username, new_password)
• Use secure password hashing (bcrypt via passlib or bcrypt). If adding a dependency, update requirements.txt.
• Use atomic file writes and simple file locking to prevent race conditions (use tempfile + os.replace for atomic write and optionally portalocker for locking). Keep dependencies minimal; implement a fallback locking mechanism if portalocker isn't available.
• Create the data directory data/ with initial files data/comments.json ({}), data/users.json ({}) and ensure they're created automatically if missing.

3. Integrate with application

• Replace current ad-hoc storage calls for comment creation, retrieval, and user register/login with the new data_store API. Update routes/controllers so that login and comment submission work against JSON storage.
• If the project uses sessions, preserve existing login/session behavior and only switch backend storage for user credentials.

4. Tests and examples

• Add unit tests for data_store functions (tests/test_data_store.py) covering creating users, authenticating, adding comments, retrieving comments, concurrent writes (if feasible).

5. Documentation

• Update README.md with instructions on the new JSON storage: location of data files, how to initialize, default file formats, and how to run tests.

6. Misc cleanup

• Remove unused files/temporary artifacts discovered during the audit.
• Add .gitignore entries for data/ if users should not commit their local data files (but leave example files like data/comments.json.example committed). Add data/comments.json.example and data/users.json.example if appropriate.

Constraints and quality requirements:

• Do not change application behavior except for storage backend and code cleanups.
• Use secure password hashing; do not store plaintext passwords.
• Ensure atomic writes to JSON files to avoid corruption with concurrent requests.
• Keep changes minimal and well-documented; add comments where important.

Deliverable: Create a pull request that implements the above: adds the data_store module, creates data/ with example JSON files, updates routes to use the new API, adds unit tests, updates README, and performs small formatting/lint fixes. The PR title should be the problem title above and description should summarize the tasks completed and any migration steps for existing installations.

---

💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.