Qt5x dev

README.md

@@ -6,7 +6,7 @@
 
 Visual intel chat analysis, planning and notification application for [EVE Online](http://www.eveonline.com). Gathers status through in-game intelligence channels on all known hostiles and presents all the data on a [dotlan](http://evemaps.dotlan.net/map/Cache#npc24) generated regional map. The map is annotated in real-time as players report intel in monitored chat channels.
 
-Vintel is written with Python 2.7, using PyQt4 for the application presentation layer, BeautifulSoup4 for SVG parsing, and Pyglet for audio playback.
+Vintel is written with Python 2.7, using PyQt5 for the application presentation layer, BeautifulSoup4 for SVG parsing, and Pyglet for audio playback.
 
 ### News
 _The current release version of Vintel [can be found here](https://github.com/Xanthos-Eve/vintel/releases). Both Mac and Windows distributions are now available for download with this release._
@@ -74,6 +74,22 @@ pyglet is used to play the sound – If it is not available the sound option wil
 https://pypi.python.org/pypi/requests
 - Six for python 3 compatibility https://pypi.python.org/pypi/six
 
+## Note for windows users
+
+The anaconda python package will come with most of what you need [download](https://www.continuum.io/downloads#windows).
+vintel compilation has been tested with 64 bit anaconda2 v4.2.0 for python 2.7 on windows 10.
+
+You will need to run this command to install the pyglet package.  If you installed anaconda for all
+users, you may need to run them in an *admin* anaconda prompt.
+
+```
+pip install pyglet
+```
+
+## Note for cygwin users
+
+pyglet doesn't currently work under cygwin.  Skip installing it or pip uninstall if it causes problems.
+
 ## Building the Vintel Standalone Package
 
  - The standalone is created using pyinstaller. All media files and the .spec-file with the configuration for pyinstaller are included in the source repo. Pyinstaller can be found here: https://github.com/pyinstaller/pyinstaller/wiki.

SETUP.md

@@ -0,0 +1,127 @@
+# Vintel Setup
+
+This document describes the settings available in the Vintel application.
+
+## Settings Dialog
+
+The settings dialog has three panes: *Quick Setup*, *Jumpbridges*, and *Chat Channels*.
+
+The *Jumpbridges* pane allows you to specify a source for jumpbridges to be 
+rendered on the maps.   You may specify either a DOTLAN jumpbridge list id or an URL to
+a file containing your own jumpbridge list.   More details on the format of the file
+can be seen in the application pane or in *Quick Setup* below.
+
+The *Chat Channels* pane allows you to specify EVE chat channels to monitor for
+intelligence reports.   You must keep this chat channel open in a tab in game
+and have *Log Chat to File* selected in EVE Settings > Chat.   Vintel monitors and
+processes these log files on your local disk.
+
+## Quick Setup
+
+The first pane of the settings allows you to paste a quick configuration JSON
+blob that may be provided by your alliance.
+
+1. **dotlan_jb_id** The id of a jumpbridge list from DOTLAN.  Include just the id,
+   i.e. from ht<i></i>tp://evemaps.dotlan.net/bridges/**XXXXxxxx** use the XXXXxxxx
+2. **jumpbridge_url** A url to a file containing your bridge list.   Each line
+   in the file should contain two systems separated by a `<->`.   i.e.
+   `HED-GP <-> 36N-HZ`
+3. **channels** A JSON list of the channels to be monitored.
+4. **kos_url** A URL to your alliances KOS server.
+5. **region_name** The default region to load on launch.  Later, the application will
+   remember the last region viewed and load that when you restart.
+6. **quick_regions** The regions listed here are automatically added to the
+   application's *Region* menu.  Any name starting with a dash will be interpreted
+   as a separator and will draw a horizontal line between entries.  Each entry
+   should have a **label** field which is how it will appear in the menu.  Each
+   may also have a **region** field which can be used to override the region name
+   used when looking up this system on DOTLAN if the label is not an exact match
+   (spaces will be automatically converted to underscores).
+
+   The **region** should match a DOTLAN url - i.e. ht<i></i>tp://evemaps.dotlan.net/map/**Catch**.
+   The **region** can also be one of the special or combined maps listed at the
+   bottom of the DOTLAN universe page.  Vintel also recognizes the custom region
+   providencecatch.
+
+This is what the Vintel defaults would look like as a quick configuration entry:
+
+```
+{
+  "dotlan_jb_id": "",
+  "jumpbridge_url": "",
+  "channels": [
+    "TheCitadel",
+    "North Provi Intel",
+    "North Catch Intel",
+    "North Querious Intel"
+  ],
+  "kos_url": "http://kos.cva-eve.org/api/",
+  "region_name": "Catch",
+  "quick_regions": [
+    {"label": "Catch"},
+    {"label": "Providence"},
+    {"label": "Querious"},
+    {"label": "---"},
+    {"label": "Provi / Catch", "region": "providencecatch"},
+    {"label": "Provi / Catch (compact)", "region": "Providence-catch"}
+  ]
+}
+```
+
+## Other Menu Settings
+
+### File > Clear Cache
+
+If you notice any weirdness with maps or avatars, you may try flushing the Vintel
+cache.  Avatars and maps will be redownloaded.
+
+### Chat > Show Chat
+
+Hide the chat history on the right side of the screen
+
+### Chat > Show Chat Avatars
+
+Do not show character portraits in the chat history.   Unfortunately selecting this
+doesn't yet make the chat window more compact.
+
+### Sound > Activate Sound
+
+This should be checked to enable sound in Vintel.
+
+### Sound > Sound Setup ...
+
+Test the sound configuration.
+
+### Sound > Spoken Notifications
+
+Doesn't work on Windows 10, what does it do?
+
+### Region
+
+Select a region from the quick region shortcuts configured in *Quick Settings*
+
+### Region > Other Region
+
+Select from a list of all known EVE regions.
+
+### Region > Custom Region ...
+
+Displays a freeform text box to enter a standard or custom region by name.
+
+### K.O.S.
+
+Activate some of the Kill On Sight processing and warnings.
+
+### Window > Always On Top
+
+Keep this window above all other windows.
+
+### Window > Frameless Main Window
+
+Remove OS decorations from the window: titlebar, menubar, ...    Once activated a *Restore Frame* button will
+be available in the upper left corner of the screen.  Click that to restore the window decorations.
+
+### Window > Transparency
+
+Set the window opacity.
+

TODO.md

@@ -0,0 +1,19 @@
+ROLLED THIS BACK FROM MainWindowUi, seems QWebEngineWidgets is not available in most PyQt5 builds
+
+```
+</widget>
+<customwidgets>
+ <customwidget>
+  <class>QWebView</class>
+  <class>QWebEngineView</class>
+  <extends>QWidget</extends>
+  <header>QtWebKit/QWebView</header>
+  <header>PyQt5/QtWebEngineWidgets/QWebEngineView</header>
+ </customwidget>
+ <customwidget>
+  <class>PanningWebView</class>
+  <extends>QWebView</extends>
+  <extends>QWebEngineView</extends>
+  <header>vi/PanningWebView</header>
+ </customwidget>
+```

quick_config_cva.json

@@ -0,0 +1,20 @@
+{
+  "dotlan_jb_id": "",
+  "jumpbridge_url": "",
+  "channels": [
+    "TheCitadel",
+    "North Provi Intel",
+    "North Catch Intel",
+    "North Querious Intel"
+  ],
+  "kos_url": "http://kos.cva-eve.org/api/",
+  "region_name": "Catch",
+  "quick_regions": [
+    {"label": "Catch"},
+    {"label": "Providence"},
+    {"label": "Querious"},
+    {"label": "---"},
+    {"label": "Provi / Catch", "region": "providencecatch"},
+    {"label": "Provi / Catch (compact)", "region": "Providence-catch"}
+  ]
+}

src/vi/PanningWebView.py

@@ -137,7 +137,7 @@ def __init__(self, parent=None):
 
     def acceptNavigationRequest(self, url, type, isMainFrame):
         if MainWindow.oldStyleWebKit:
-            return super(PanningWebView, self).acceptNavigationRequest(url, type, isMainFrame)
+            return super(VintelSvgPage, self).acceptNavigationRequest(url, type, isMainFrame)
         else:
             if type == QWebEnginePage.NavigationTypeLinkClicked:
                 self.view().mapLinkClicked.emit(url)

src/vi/cache/cache.py

@@ -21,6 +21,7 @@
 import threading
 import time
 import six
+import sys
 if six.PY2:
     def to_blob(x):
         return buffer(str(x))
@@ -30,7 +31,7 @@ def from_blob(x):
     def to_blob(x):
         return x
     def from_blob(x):
-        return x
+        return x[0][0]
 from vi.cache.dbstructure import updateDatabase
 
 
@@ -73,7 +74,7 @@ def checkVersion(self):
         updateDatabase(version, self.con)
 
     def putIntoCache(self, key, value, maxAge=60 * 60 * 24 * 3):
-        """ Putting something in the cache maxAge is maximum age in seconds
+        """ Putting something in the cache maxAge is maximum age in seconds, default is 3 days
         """
         with Cache.SQLITE_WRITE_LOCK:
             query = "DELETE FROM cache WHERE key = ?"
@@ -96,6 +97,32 @@ def getFromCache(self, key, outdated=False):
         else:
             return founds[0][1]
 
+    def getConfigValue(self, key):
+        """ Retrieve a config value from cache that never expires
+        """
+        return self.getFromCache(key, True)
+
+    def saveConfigValue(self, key, value):
+        """ Save a config value to cache that never expires
+        """
+        return self.putIntoCache(key, value, sys.maxsize)
+
+    def deleteFromCache(self, key):
+        """ Deleteing from cache
+        """
+        with Cache.SQLITE_WRITE_LOCK:
+            query = "DELETE FROM cache WHERE key = ?"
+            self.con.execute(query, (key,))
+            self.con.commit()
+
+    def flush(self):
+        """ Flush non config items from cache
+        """
+        with Cache.SQLITE_WRITE_LOCK:
+            query = "DELETE FROM cache WHERE maxAge < ?"
+            self.con.execute(query, (sys.maxsize,))
+            self.con.commit()
+
     def putPlayerName(self, name, status):
         """ Putting a playername into the cache
         """
@@ -136,8 +163,8 @@ def getAvatar(self, name):
         if len(founds) == 0:
             return None
         else:
-            # dats is buffer, we convert it back to str
-            data = from_blob(founds[0][0])
+            # data is buffer, we convert it back to str
+            data = from_blob(founds)
             return data
 
     def removeAvatar(self, name):

src/vi/chatparser/chatparser.py

@@ -20,13 +20,15 @@
 import datetime
 import os
 import time
+import logging
 import six
 
 from PyQt5.QtWidgets import QMessageBox
 if six.PY2:
     from io import open
 from bs4 import BeautifulSoup
-from vi import states
+from vi import states, evegate
+
 
 
 from .parser_functions import parseStatus
@@ -40,12 +42,16 @@ class ChatParser(object):
     """ ChatParser will analyze every new line that was found inside the Chatlogs.
     """
 
-    def __init__(self, path, rooms, systems):
+    def HEADER_SIZE(self): return 13
+    def REPLAY_SIZE(self): return 500                   # retreive up to 500 lines from a logfile when starting up
+
+    def __init__(self, path, rooms, allSystems, messageExpirySecs):
         """ path = the path with the logs
             rooms = the rooms to parse"""
+        self.messageExpirySecs = messageExpirySecs
         self.path = path  # the path with the chatlog
         self.rooms = rooms  # the rooms to watch (excl. local)
-        self.systems = systems  # the known systems as dict name: system
+        self.allSystems = allSystems  # the known systems as dict name: system
         self.fileData = {}  # informations about the files in the directory
         self.knownMessages = []  # message we allready analyzed
         self.locations = {}  # informations about the location of a char
@@ -103,6 +109,9 @@ def _lineToMessage(self, line, roomname):
             timestamp = datetime.datetime.strptime(timeStr, "%Y.%m.%d %H:%M:%S")
         except ValueError:
             return None
+        if (evegate.currentEveTime() - timestamp).total_seconds() > self.messageExpirySecs:
+            # logging.debug('ignoring message from ' + timeStr + " || " + str((evegate.currentEveTime() - timestamp).total_seconds()) )
+            return None
         # finding the username of the poster
         userEnds = line.find(">")
         username = line[timeEnds + 1:userEnds].strip()
@@ -136,7 +145,7 @@ def _lineToMessage(self, line, roomname):
             continue
         while parseUrls(rtext):
             continue
-        while parseSystems(self.systems, rtext, systems):
+        while parseSystems(self.allSystems, rtext, systems):
             continue
         parsedStatus = parseStatus(rtext)
         status = parsedStatus if parsedStatus is not None else states.ALARM
@@ -154,9 +163,6 @@ def _lineToMessage(self, line, roomname):
         message.message = six.text_type(rtext)
         message.status = status
         self.knownMessages.append(message)
-        if systems:
-            for system in systems:
-                system.messages.append(message)
         return message
 
     def _parseLocal(self, path, line):
@@ -205,7 +211,7 @@ def fileModified(self, path):
         roomname = filename[:-20]
         if path not in self.fileData:
             # seems eve created a new file. New Files have 12 lines header
-            self.fileData[path] = {"lines": 13}
+            self.fileData[path] = {"lines": self.HEADER_SIZE()}
         oldLength = self.fileData[path]["lines"]
         lines = self.addFile(path)
         if path in self.ignoredPaths:
@@ -222,6 +228,22 @@ def fileModified(self, path):
                     messages.append(message)
         return messages
 
+    def rewind(self):
+        """On region change or startup, rewind files to replay info from logfiles"""
+        for path in self.fileData:
+            newsize = self.fileData[path]["lines"] - self.REPLAY_SIZE()
+            if newsize < self.HEADER_SIZE():
+                newsize = self.HEADER_SIZE()
+            logging.info("Rewinding %d lines from %s" % (self.fileData[path]["lines"]-newsize, path))
+            self.fileData[path]["lines"] = newsize
+        return self.fileData
+
+    def expire(self):
+        for m in self.knownMessages:
+            if (evegate.currentEveTime() - m.timestamp).total_seconds() > self.messageExpirySecs:
+                self.knownMessages.pop(0)
+            else:
+                break
 
 class Message(object):
     def __init__(self, room, message, timestamp, user, systems, upperText, plainText="", status=states.ALARM):
@@ -235,6 +257,7 @@ def __init__(self, room, message, timestamp, user, systems, upperText, plainText
         self.plainText = plainText  # plain text of the message, as posted
         # if you add the message to a widget, please add it to widgets
         self.widgets = []
+        self.isOld = False
 
     def __key(self):
         return (self.room, self.plainText, self.timestamp, self.user)