Skip to content

Nedja995/twint_kibana

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

3 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

TWINT - More practical (and optimized) use with Elasticsearch and Kibana

See also Twint Flask-Celery Server for http server

Table of Contents

  1. Analyze keywords tutorial
  2. Share dashboard
  3. Optimized twint use
  4. Tips

Requirements

  • Python3, Twint
  • Elasticsearch
  • Kibana

Analyze keywords tutorial

  1. Create ES index with index-tweets.json

  2. Gather tweets containing keyword (3 ways)

  1. Create Kibana Index Pattern

  2. (optional) Add scripted field shared_url_base to Kibana Index Pattern using painless_url_base.txt

  3. Generate visualization and import in Kibana Saved Objects
    python3 elasticsearch/generate_visualizations.py <Kibana index patter id> -n <optional (index)name>

Share dashboard

Optimized twint use

  • New parametars: -rd Request Days and -mi Maximum Instances to run.

  • python3 utils/otwint.py -s "<keyword>" --since 2019-1-1 --until 2019-2-1 -es localhost:9200 -it "<es index name>" -rd 1 -mi 4

Tips

  • Enable regex. In /etc/elasticsearch/elasticsearch.yml add line script.painless.regex.enabled: true

TODO

  • user_created_at (python script)
  • resolve short urls (bit.ly,..) (python script)

TODO: Automatize

After user enter parametars keyword, since datetime, until datetime do in backround

  1. create ES index
  2. create Kibana index pattern Ref Ref 2
  3. get new Kibana index id
  4. generate visualizations
  5. import visualization to Kibana Saved Objects
  6. Kibana dashboard ready

About

More practical use of Twint with ES and Kibana.

Topics

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published