stracker 3.5.1

The one or the other might have heard about the new EU privacy policy (GDPR). We already have had some discussions here: https://www.assettocorsa.net/forum/...olicy-a-show-stopper-for-tracking-apps.49496/. I believe that this is or at least might be a problem for stracker. There is a lot of uncertainty out there, and I am not willing to be a "beta tester" of this new law for the sake of a computer game.

I hope you understand that I take no responsibility for data processing on 3rd party servers, even if the data processing is performed by a software which was mostly written by me. You as server admins are fully responsible for the software running on your servers and you have to judge for yourself if this is or is not a legal issue in your specific usecase.

Nevertheless, I have made some changes to stracker, so that

1. stracker does not store plain steam GUID's anymore in the database. I don't know whether steam ID's are personal data, nor do I know whether the sha256 hashes of steam ID's are personal data. There might be more personal data stored in the stracker database. Judge for yourself please.
2. users can anonymize their data by using the new chat command "/st anonymize on". This deletes also their chat messages from the database. I don't know if this is enough to provide users the right to delete their personal data or not. You have to judge that for yourself.
3. admin can choose to block those users from entering the server via stracker auth

Informing the user is under responsibility of the server admins.

I'm not saying that running stracker 3.5.x is enough to comply to the GDPR. I'm not even saying that the changes made in 3.5.x have the purpose to comply to GDPR.

Known Issues of stracker 3.5.x:

• Entry list generation is using wrong GUID's now (because they are not stored in DB anymore and so the information is not available)
• guids_based_on_driver_names = True may not work correctly anymore due to changes in GUID handling. I suggest to use stracker 3.4.x for these setups

This is the full change log for stracker 3.5.1:

core:

• add config option HTTP_CONFIG/auth_ban_anonymized_players (default: False)
• add config options WELCOME_MSG/line[4..6] with a default message about tracking. Please see the general remarks about the GDPR above.
  
• remove options HTTP_CONFIG/enable_paypal_link and BLACKLIST/blacklist_file
• upgrade database to version 24, convert steam ID's to sha256 hashes
• anonymize steam ID's saved in the server database by sha256 hashes
• provide a user command to anonymize his personal data. Results in deletion of his chat messages, and his name will be replaced with an anonymous name, which is not updated anymore until the user chooses to exit this mode. Admins can choose to automatically block those players
• Updates in README.txt, mention the EU GDPR dilemma and make sure that the server admin takes responsibility
• avoid creating player entries directly after a player has been joining
• send welcome messages always over chat, so that they are better visible to a driver even when he is running ptracker

http:

• fix sort order of drivers in drrivers page for postgres databases
• changes in some of the http templates

Change Log:

• fix possible exception when rendering driver details page
• remove total km driven because of long computation times
• deprecate the BLACKLIST/blacklist_file config entry. It is recommended to leave that one empty and use the stracker authentication system instead. (Users reported that sometimes GUIDs are not removed correctly from the blacklist file)

Change Log:

• New config option STRACKER_CONFIG/ac_server_working_dir for setting the working directory of the AC server process. You only need to set it, if you are using an unusual directory structure; you will get warnings in your logs in this case
• possible fix for incorrect race results: stracker now reads the json results and adapts the results accordingly (see also config option STRACKER_CONFIG/ac_server_working_dir)
• Changes from AC forum user @maxator: option for using https protocol + update to new cherrypy version
• experimental fix for the rejoining problems in quali and practice sessions
• a little bit more statistics in the player details page

Changes

• save ESC key press information correctly in DB again
• non-ptracker player's first lap is invalidated now

New stable version 3.3.5 adds mainly support for timed races:

• Changes in stracker ini file
  Section Option name Default value Documentation
  STRACKER_CONFIG guids_based_on_driver_names False you normally want to leave that at the default (False). Use case for this is an environment where the same steam account is used by different drivers.
• New Database Scheme Version 23: add cache table in db to cache MR grades
• send MR grades to connected ptracker instances for display purpose
• make timed races work in stracker

Required update to fix the lap count issue (and more).

Changes compared to 3.1.x (better formatted change logs are available here: http://www.n-e-y-s.de/stracker_doc#auto_chapter_20)

Changes in stracker core

• Changes in stracker ini file
  
  Section Option name Default value Documentation
  SWEAR_FILTER warning Please be polite and do not swear in the chat. You will be %(swear_action)s from the server after receiving %(num_warnings_left)d more warnings message sent to player after a swear detection
  SWEAR_FILTER num_warnings 3 Specify the number of warnings issued before the player is going to be kicked
  SWEAR_FILTER action none Valid values are "none", "kick" and "ban".
  SWEAR_FILTER ban_duration 30 Number of days the player shall be banned (if action is "ban").
  SWEAR_FILTER swear_file bad_words.txt Specify a file with bad words (one in each line). See https://github.com/shutterstock/List-of-Dirty-Naughty-Obscene-and-Otherwise-Bad-Words for examples.
  HTTP_CONFIG enable_flattr_link
  Removed in favor of paypal.
  HTTP_CONFIG inverse_navbar False set to true to get the navbar inverted (i.e., dark instead of bright)
  HTTP_CONFIG velocity_unit kmh Valid values are "kmh" or "mph".
  HTTP_CONFIG temperature_unit degc Valid values are "degc" or "degf".
  HTTP_CONFIG lap_times_add_columns valid+aids+laps+date Additional columns to be displayed in LapTimes table (seperated by a + sign). Columns can be "valid", "aids", "laps", "date", "grip", "cuts", "collisions", "tyres", "temps" and "ballast". Note that too many displayed columns might cause problems on some browsers.
• New Database Scheme Version 22
  • add column MessagesDisabled to table Players
  • add column ballast to table Lap
  • seperate the HistoryInfo column from the Lap table, resulting in much better performance of best lap time queries
  • add ChatHistory table
• add point-to-point (A2B) track support with special handling for ks_nordschleife-touristenfahrten
• add a configurable swear filter leading to kick or ban actions after a specified number of warnings
• players without ptracker can now decide to not receive any messages from stracker if these messages get too annoying. Decision will be stored in database
• Avoid 100% cpu load if the sqlite database is locked for some reason
• add /st whisper command (usage: "/st whisper guid <guid> <text>" or "/st whisper id <carid> <text>")
• add /st broadcast command
• rename commands kickban_guid and kickban_id to kickban guid and kickban id
• change stracker/ptracker protocol for new ptracker stat display, ballast and fuel load
• display the player's car in the enter message when multiple cars are allowed in the server
• fix inconsistent timestamps (the timezone should be respected now correctly)
• fix lap stat time filtering
• fix qualify classification to match AC style
• fix ptracker information of very last lap of qualify session
• Switch the .exe generation tooling from py2exe to pyinstaller on windows systems
• fix error when compressing the database (introduced in 3.2.0)
• fix "unability to start" when the blacklist_file in stracker.ini is set.
• add return to pits heuristics (needed for A2B tracks)
• fix db compression errors
• if db compression is enabled, compress immediately when starting stracker instead of waiting for the periodic timer
• fix possible hangups while exiting
• remove print() debugging statements spamming the console window
• fix log errors when trying to send chat messages to disconnected players
• postres: better error message when trying to migrate to non-empty database
• postres: fix migration with some legacy session tables (duration too large)

Changes in stracker http

• admins can now whisper, kick and ban directly from classification view in live map
• add non-existing tracks and cars in stracker-packager data to database, rather than throw them away (=less upload frequency)
• configurable columns for lap stat table
• add option for inverse navbars
• display chat in live map
• add session management and broadcast chat possibilities (when logged in as admin) in live map
• display not connected players in gray in the classification section of live map
• do not display the positions of unconnected players anymore on the map anymore in live map
• include custom.css file in all html pages, so the css styles can be easily customized
• remove flattr donation possibility
• change filtering in multiselects to case insensitive
• add log display for chat entries (for admins only)
• fix (C)orvette display and similar issues (the brand name is now required to be at least 3 characters)
• possibility for programmers to change http output by making the templates editable. (Note: I can't give support about how to modify these files, you do this adventure for your self.)
• fix comparison lap choices for multi-car sessions
• reduce stracker's memory usage by not holding all maps and car badges in memory but only the ones needed
• changes to templates for supporting the in-game browser
• display available car badges and track maps correctly for local databases (they are added automatically to the database)
• update to pygal version 2.1.1 and reduce the size of the lap time comparisons.
• add fuel display for sessions which have already ended (you won't be able to see the fuel loads for cars in the current session)
• fix problems when dreamweaver is installed on windows
• fix typo "unknwon" (thanks for reporting, core3tx)

Changes in stracker-packager

• support tracks with main layout and config layouts at the same time (e.g. the mod for increasing the number of slots for KS tracks)
• when an error occurs, display the message and leave the command window open
• better detection of AC install dir
• fix errors when having files in the content/cars or content/tracks directories

Uploaded experimental versions 3.2.8 of ptracker and stracker to http://n-e-y-s.de.

stracker changes:

• http: fix problems when dreamweaver is installed on windows

Uploaded experimental versions 3.2.7 of stracker and ptracker to http://n-e-y-s.de. This is now the release candidate.

Changes:

stracker:

• core: update stracker/ptracker protocol (backwards compatible) to communicate the fuel load to stracker
• core: display the player's car in the enter message when multiple cars are allowed in the server
• http: add fuel display for sessions which have already ended (you won't be able to see the fuel loads for cars in the current session)

Uploaded experimental versions 3.2.6 of stracker and ptracker to http://n-e-y-s.de. I think we are finally getting somewhere close to stable ...

Change Logs:

stracker

• fix db compression (really really)
• remove print() debugging statements spamming the console window

Uploaded new experimental versions 3.2.5 of ptracker and stracker to http://n-e-y-s.de. Unexperienced users shall stay with the latest stable 3.1.x versions.

Thanks to all of you reporting issues and bugs of these experimental versions. For this version, special thanks to @QC.

Notes:

• Versions 3.2.0 to 3.2.4 have problems with the stracker/ptracker protocol backwards compatibility and they are not able to connect with these new versions and also not with 3.1.x.
  • All users of these versions are encouraged to update to 3.2.5

Changes:

• core: changes in stracker/ptracker protocol to ensure compatibility with ptracker 3.1.x. ptracker versions 3.2.0 to 3.2.4 are incompatible with this version (sorry)
• core: fix db compression errors
• core: if db compression is enabled, compress immediately when starting stracker instead of waiting for the periodic timer
• core: fix possible hangups while exiting
• postres: better error message when trying to migrate to non-empty database
• postres: fix migration with some legacy session tables (duration too large)
• http: fix admin link urls
• http: (hopefully) fix wrong display of car badges
• http: fix vmax display
• stracker-packager: better detection of AC install dir
• stracker-packager: fix errors when having files in the content/cars or content/tracks directories

Uploaded new experimental version 3.2.4 for stracker and ptracker. As usual, experimental versions can be downloaded from http://n-e-y-s.de

Change Log:

stracker

• http: Potential fix for the disappearing car badges and map displays.
• http: update to pygal version 2.1.1 and reduce the size of the lap time comparisons.

Here we go, experimental ptracker and stracker version 3.2.3. As usual you can download experimental versions from http://n-e-y-s.de, unexperienced users shall stay with the stable versions. I still expect some bugfix versions needed for 3.2.x.

Highlights:

• support for point to point tracks (A2B) in both stracker and ptracker

Notes:

• I've set up a server running nordschleife tourist (NEYS 3). If you want to help, it'd be cool if you could do some laps in it to have stracker's A2B track support tested.

Changes:

• Changes in stracker ini file
  Section Option name Default value Documentation
  HTTP_CONFIGvelocity_unit kmh Valid values are "kmh" or "mph".
  HTTP_CONFIGtemperature_unit degc Valid values are "degc" or "degf".
  HTTP_CONFIGlap_times_add_columns valid+aids+laps+date Additional columns to be displayed in LapTimes table (seperated by a + sign). Columns can be "valid", "aids", "laps", "date", "grip", "cuts", "collisions", "tyres", "temps" and "ballast". Note that too many displayed columns might cause problems on some browsers.
• core: add point-to-point (A2B) track support with special handling for ks_nordschleife-touristenfahrten
• core: add return to pits heuristics (needed for A2B tracks)
• core: fix possible connection timeouts occuring during remote in-game-browsing (backwards compatible, works with ptracker 3.2.3)
• core: add profiling information of db queries in debug logging
• http: add unit configurations also to stracker (HTTP_CONFIG/velocity_unit+temperature_unit)
• http: reduce stracker's memory usage by not holding all maps and car badges in memory but only the ones needed
• http: changes to templates for supporting the in-game browser
• http: display available car badges and track maps correctly for local databases (they are added automatically to the database)
• http: add unit configurations to stracker.ini (HTTP_CONFIG/velocity_unit+temperature_unit)
• http: additional optional column vmax in lap stat page (HTTP_CONFIG/lap_times_add_columns)
• stracker-packager: support tracks with main layout and config layouts at the same time (e.g. the mod for increasing the number of slots for KS tracks)
• stracker-packager: when an error occurs, display the message and leave the command window open

stracker 3.2.2-experimental now available for download

ChangeLog: fix for unability to start with blacklist_file set

stracker, version 3.2.1-experimental are available for download at http://n-e-y-s.de/download

Change Log:

• core: change stracker/ptracker protocol for new ptracker stat display
• core: fix error when compressing the database (introduced in 3.2.0)
• http: fix comparison lap choices for multi-car sessions

Uploaded stracker 3.1.7 experimental to http://n-e-y-s.de/download. Please only use this version if you are prepared for some instabilities and bugs. Otherwise stick with the stable version. There are quite a lot new features and also some bugfixes. Here is the change log (see also http://n-e-y-s.de/stracker_doc#auto_chapter_21 for a better readable version):

• Changes in stracker ini file
  • SWEAR_FILTER warning Please be polite and do not swear in the chat. You will be %(swear_action)s from the server after receiving %(num_warnings_left)d more warnings message sent to player after a swear detection
  • SWEAR_FILTER num_warnings 3Specify the number of warnings issued before the player is going to be kicked
  • SWEAR_FILTER actionnoneValid values are "none", "kick" and "ban".
  • SWEAR_FILTER ban_duration30Number of days the player shall be banned (if action is "ban").
  • SWEAR_FILTER swear_filebad_words.txtSpecify a file with bad words (one in each line). See https://github.com/shutterstock/List-of-Dirty-Naughty-Obscene-and-Otherwise-Bad-Words for examples.
  • HTTP_CONFIG enable_flattr_link Removed in favor of paypal.
  • HTTP_CONFIG lap_times_add_columnsvalid+aids+laps+dateAdditional columns to be displayed in LapTimes table (seperated by a + sign). Columns can be "valid", "aids", "laps", "date", "grip", "cuts", "collisions", "tyres", "temps" and "ballast". Note that too many displayed columns might cause problems on some browsers.
  • HTTP_CONFIG inverse_navbarFalseset to true to get the navbar inverted (i.e., dark instead of bright)
• New Database Scheme Version 22
  • add column MessagesDisabled to table Players
  • add column ballast to table Lap
  • seperate the HistoryInfo column from the Lap table, resulting in much better performance of best lap time queries
  • add ChatHistory table
• core: Avoid 100% cpu load if the sqlite database is locked for some reason
• core: rename commands kickban_guid and kickban_id to kickban guid and kickban id
• core: add /st whisper command (usage: "/st whisper guid <guid> <text>" or "/st whisper id <carid> <text>")
• core: add /st broadcast command
• core: add a configurable swear filter leading to kick or ban actions after a specified number of warnings
• core: players without ptracker can now decide to not receive any messages from stracker if these messages get too annoying. Decision will be stored in database
• core: fix inconsistent timestamps (the timezone should be respected now correctly)
• core: fix lap stat time filtering
• core: fix qualify classification to match AC style
• core: fix ptracker information of very last lap of qualify session
• core: Switch the .exe generation tooling from py2exe to pyinstaller on windows systems
• http: display chat chat in live map
• http: display not connected players in gray in the classification section of live map
• http: do not display the positions of unconnected players anymore on the map anymore in live map
• http: admins can now whisper, kick and ban directly from classification view in live map
• http: add session management and broadcast chat possibilities (when logged in as admin) in live map
• http: add non-existing tracks and cars in stracker-packager data to database, rather than throw them away (=less upload frequency)
• http: include custom.css file in all html pages, so the css styles can be easily customized
• http: configurable columns for lap stat table
• http: remove flattr donation possibility
• http: add log display for chat entries (for admins only)
• http: add option for inverse navbars
• http: change filtering in multiselects to case insensitive
• http: fix (C)orvette display and similar issues (the brand name is now required to be at least 3 characters)
• http: add custom.css file
• http: possibility for programmers to change http output by making the templates editable. (Note: I can't give support about how to modify these files, you do this adventure for your self.)

Change Log, compared to version 3.0.x (see also http://n-e-y-s.de/stracker_doc):

Alternative download page: http://n-e-y-s.de/download

• Changes in stracker ini file
  
  Section Option name Default value Documentation
  STRACKER_CONFIG append_log_file False Set to 1, if you want to append to log files rather than overwriting them. Only meaningful with external log file rotation system.
  SESSION_MANAGEMENT race_over_strategy none What to do when the race is over and no player is actively racing. Valid values are: "none" or "skip".
  SESSION_MANAGEMENT wait_secs_before_skip 15 Number of seconds to wait before the session skip is executed (if race_over_strategy is set to "skip")
  HTTP_CONFIG auth_log_file
  Set to a file to be used for logging http authentication requests. Useful to prevent attacks with external program (e.g., fail2ban).
  HTTP_CONFIG enable_paypal_link True Enable paypal link for letting users donate to the author. If you do not like that, switch it off.
  HTTP_CONFIG max_streaming_clients 10 Maximum number of streaming clients (LiveMap/Log users) allowed to connect to this server in parallel. The number of threads allocated for http serving will be max(10, max_streaming_clients + 5)
  LAP_VALID_CHECKS ptrackerAllowedTyresOut -1 if -1: use server penalty setting, if available, otherwise use 2. All other values are passed to ptracker.
  STRACER_CONFIG keep_alive_ptracker_conns True If False (not recommended), the TCP keep_alive option will be disabled (that was the behaviour pre 3.1.7). Note that users might get unexpected ptracker/stracker disconnections when this option is set to False.
• replace pysqlite3 module by apsw (big change - I hope this will be working!)
  • this should be more stable in database migration, even on multiserver setups
• server reports now the combo ranking in case of a personal best / server best in addition to the car specific ranking
• add session management feature
  • stracker can now be configured to automatically skip race sessions, after everyone has finished (default: off)
  • add config option SESSION_MANAGEMENT/wait_secs_before_skip (default: 15 seconds)
  • add admin commands "/st session rlaps", "/st session qtime" and "/st session ptime" for setting number of laps / time of sessions in minutes
    • player has to be in group "admins" to be able to use them
• add config option LAP_VALID_CHECKS/numberOfTyresOut
  • if set to -1 (default) it uses the AC server's penalty setting, if valid, otherwise it uses 2
  • if set to a number between 0 and 3, it configures ptracker for the allowed number of tyres out for valid laps
• do not count laps directly after session changes
• be more verbose about database migration
• better error reporting for port configuration issues
• experimental workaround for AC 1.3.3 linux server
• new stracker/ptracker protocol 10 (minor changes)
• finish session is now called when the ac signal arrives or when the ac server is detected to be down
• fix errors with log_requests option
• suppress warnings and debug messages from udp plugin API when server is down
• fix for A->B track caused division by zero
• fix logging of http server related warnings and errors (has been broken since introducing the log_requets option)
• fix connection losses for no reasons
• database version 21: add combo information to session
• http: add live view with map/ranking
• http: reworked statistics page
• http: more possibilities to filter lap times
• http: possibility to select comparison laps manually in lap details view
• http: add a log view in the admin section with some filtering options
• http: in case of multiple car sessions, the default lapstat filters are now correct even if not all cars have been used in last session
• http: better display of grip level in lap details
• http: fix parsing of json UI files by usage of a new parser module (simplejson instead of python's json)
• http: add admin button to invalidate all selected laps in statistics page
• http: add optional paypal link in the footer (you can disable that in stracker.ini: HTTP_CONFIG/enable_paypal_link)
• http: fix multiserver log file display
• http: fix logging of http server related warnings and errors (has been broken since introducing the log_requets option)
• acauth: add more criteria for authorization
• acauth: add proxy feature for proxying e.g. minorating including a timeout

Uploaded stracker 3.1.7 experimental to http://n-e-y-s.de/download. This one has high chances to be declared stable during this week.

Change Log:

• fix logging of http server related warnings and errors (has been broken since introducing the log_requets option)
• new option STRACER_CONFIG/keep_alive_ptracker_conns (default: True)
• fix connection losses for no reasons
• fix command /st session qtime and ptime

Uploaded stracker 3.1.6, experimental version to http://n-e-y-s.de/download. Hopefully finally the candidate for the stable version... Please try it out as soon as possible and report any issues (if any ...).

Changes

• fix statistics page in case stracker has been backdated to 3.0.x after db migration
• update links to project pages/FAQ in stracker main page

Uploaded ptracker/stracker versions 3.1.5. This should be the last experimental before going stable, so if you have a chance, it would be nice you could give it a go. As usual, experimental downloads are here: http://n-e-y-s.de/download

Changes:
ptracker:

• fix double entry about fuel prediction display in config dialog
• fix sending setup sender name

stracker:

• fix possible error in udp plugin plausibility check (RuntimeError: dictionary changed size during iteration)
• new option HTTP_CONFIG/max_streaming_clients
• slightly optimize lap stat query statements
• acauth: introduce caching to the acauth feature needed to avoid server lag
• acauth: automatically add '&' or '?' to the andurl1..5
• http: fix streaming of too many clients
• http: show admin ban options even if no blacklist.txt file is configured
• http: fix wrong admin login failure reports for re-authenticate requests
• http: fix player details page when running sqlite3 databases
• http: fix auto-scroll of log display in case of no log changes
• linux: provide SO_ALLOWREUSE in socket options, so that the 4 minutes pause between stracker stop and starts is gone