Meeting December 2019 (Programming)

Guidance

• Follow the process outlined in 'implementing a disease module'

  • Have a basic test file to run your code
  • Develop the model incrementally, adding complexity gradually
    • Catch problems early
    • Easier for us to help!

• Master should be merged into your branches soon after PRs are merged

  • Notification in the Slack #programming channel
  • Prevents complicated conflicts
  • Keeps your branch up-to-date

• Open draft PRs on Github

  • Can use the collaboration tools but indicates work-in-progress

New utility functions and requests for me

• tlo.util.transition_states

  • Takes a single Dataframe column of states and transition probability matrix (Dataframe)
  • Returns a new column with transitioned states
  • Example on the wiki

• tlo.util.nested_to_record

  • A flattened dictionary representation of a Dataframe
  • e.g.

            Name   Region     Username
    1  Nathaniel  Midwest      nzburke
    2  Elisabeth    South     ewfoster
    3     Briana  Midwest  bclancaster
    4    Estella     West     elpotter
    5     Lamont    South      llwoods
  

  becomes

   {'First Name_1': 'Nathaniel',
    'First Name_2': 'Elisabeth',
    'First Name_3': 'Briana',
    'First Name_4': 'Estella',
    'First Name_5': 'Lamont',
    'Region_1': 'Midwest',
    'Region_2': 'South',
    'Region_3': 'Midwest',
    'Region_4': 'West',
    'Region_5': 'South',
    'User Name_1': 'nzburke',
    'User Name_2': 'ewfoster',
    'User Name_3': 'bclancaster',
    'User Name_4': 'elpotter',
    'User Name_5': 'llwoods'}
  

  • Can be used for logging

Updates to wiki and PR checklists

• Installation and setup guide
  • Still need a Windows version!
• Phase 4 & 5 from the checklist for developing a disease module
• Pre-PR checklist
  • We're figuring out the tooling on Windows!

Issues outstanding

• Improve logging using a TLO-specific logging module

  • Handles setting up the logging of TLO
  • One-liners to configure output
    • e.g. turn off, save to file etc.
  • Deal with strange output that causes problems downstream e.g. nan
  • Enforce documentation of log lines
  • Includes improving the parsing of logs
    • Filtering the log lines when parsing
    • Using a faster implementation of building dataframes from log lines

• Performance

  • Health System is the bottleneck
    • Continuing to profile and refactor
  • Over-allocating rows in the population dataframe
    • Essential that models only work on is_alive individuals!
  • The more frequent an event, the more to worry about the "work" in each call

Run Management system proposal