Refactor fetching & formatting

[ValueRaider]

With a change in #1754, a path is visible for allowing users to specify which Yahoo modules to fetch. But this needs a refactor, which I want to combine with separating fetch from format so users can get raw data with consistent API.

This discussion is to plan, initially the Git branch structure. I propose a new branch e.g. refactor, created off dev, and contributors submit pull requests to refactor. Then when ready, it becomes version 0.3.

@bot-unit

[bot-unit]

Current endpoints that i known:

    'crumb': '/v1/test/getcrumb',  # crumb for requests
    'spark': '/v7/finance/spark',  # quick info with close price data
    'quote_type': '/v1/finance/quoteType/',  # type of quote
    'quote': '/v7/finance/quote',  # quote current data
    'quote_summary': '/v10/finance/quoteSummary/',  # expanded quote data, depends on modules in query
    'chart': '/v8/finance/chart/',  # price data
    'download': '/v7/finance/download/',  # download price data
    'options': '/v7/finance/options/',  # options data
    'search': '/v1/finance/search',  # search quotes by text, answer is quotes and news
    'recommendations': '/v6/finance/recommendationsbysymbol/',  # recommendations other symbols by symbol

Also we have this endpoint:
https://query1.finance.yahoo.com/ws/fundamentals-timeseries/v1/finance/timeseries/

For example

[ValueRaider]

Seems sensible to have one fetcher per endpoint. Can go in new subfolder endpoints/, advantage of allowing incremental transition from scrapers/.

[bot-unit]

I think we don't need subfolder for endpoints. One class with methods for every endpoint is enough. So we will have one class for session management (and now we can have two classes for sync and async sessions). One as request builder (endpoints) that use session object fro request. And one or several for data parsing and manipulations. And also one entry point as current Ticker for compatibility.

[bot-unit]

All working modules of quote summary:

    "summaryProfile",  # contains general information about the company
    "summaryDetail",  # prices + volume + market cap + etc
    "assetProfile",  # summaryProfile + company officers
    "price",  # current prices
    "quoteType",  # quoteType
    "esgScores",  # Environmental, social, and governance (ESG) scores, sustainability and ethical performance of companies
    "incomeStatementHistory",
    "incomeStatementHistoryQuarterly",
    "balanceSheetHistory",
    "balanceSheetHistoryQuarterly",
    "cashFlowStatementHistory",
    "cashFlowStatementHistoryQuarterly",
    "defaultKeyStatistics",  # KPIs (PE, enterprise value, EPS, EBITA, and more)
    "financialData",  # Financial KPIs (revenue, gross margins, operating cash flow, free cash flow, and more)
    "calendarEvents",  # future earnings date
    "secFilings",  # SEC filings, such as 10K and 10Q reports
    "upgradeDowngradeHistory",  # upgrades and downgrades that analysts have given a company's stock
    "institutionOwnership",  # institutional ownership, holders and shares outstanding
    "fundOwnership",  # mutual fund ownership, holders and shares outstanding
    "majorDirectHolders",
    "majorHoldersBreakdown",
    "insiderTransactions",  # insider transactions, such as the number of shares bought and sold by company executives
    "insiderHolders",  # insider holders, such as the number of shares held by company executives
    "netSharePurchaseActivity",  # net share purchase activity, such as the number of shares bought and sold by company executives
    "earnings",  # earnings history
    "earningsHistory",
    "earningsTrend",  # earnings trend
    "industryTrend",
    "indexTrend",
    "sectorTrend",
    "recommendationTrend",
    "futuresChain",  # for futures
        "topHoldings",  # for ETFs
        "fundPerformance", # for ETFs
        "fundProfile", # for ETFs

[ValueRaider]

This thread can be for organising branches. I've created branch refactor from dev.

Suggestion for sub-branch names is refactor-...-

[ValueRaider]

I've deleted branch, will recreate when activity calms down.

[ValueRaider]

First refactor branch: moving bulky price history + repair into new file #1849

[ValueRaider]

Done.