DeveLanCacheUI_Backend

A UI for Lan Cache.

** Important Upgrade Guide**

As of version 1.0.57 you need to migrate to a new Docker-Compose.yml. Here's how:

Note: Also add the DNS line if you errors in the container logs that it can't get the manifests due to a 401 unauthorized

Screenshots

Can show Steam Games that were downloaded / cached:

Can show other Download events:

Can show client / service stats:

How to run this

Steps:

1. Create/mount the relevant directories
2. Update the docker-compose.yml with your own URL's / paths. Changing the TZ and LANG only impact the way the container logs. (E.g. 23/04/2023 instead of 04/23/2023)
3. Run the docker-compose file
4. Copy paste the app-depot-output.csv file in the mounted depotdir. This will automatically fill the database with all Depot => App mappings (Grab here: https://github.com/devedse/DeveLanCacheUI_SteamDepotFinder_Runner/releases)
5. Github Sponsor me 5k euro's kthnx
6. Profit

Minimal Setup

This setup is meant to work and change as little configuration values as possible. Only change the host part of the volumes and the DNS.

docker-compose.yml:

version: '3'

services:
  develancacheui_backend:
    image: devedse/develancacheui_backend:latest
    restart: unless-stopped
    ports:
      - '7301:80'
    environment:
      - TZ=Europe/Amsterdam
      - LANG=en_GB.UTF-8
    volumes:
      - "/home/pi/dockercomposers/develancacheui/backend:/var/develancacheuidata"
      - "/mnt/mynas/DockerComposers/lancache/logs:/var/develancacheui/lancachelogs:ro"
    dns:
      - 10.88.20.254
  develancacheui_frontend:
    image: devedse/develancacheui_frontend:latest
    restart: unless-stopped
    ports:
      - '7302:80'
    environment:
      - BACKENDURL=http://10.88.20.2:7301 #iclude http/https here

Advanced setup

It's possible that you want to configure more, here's an example for that.

docker-compose.yml:

version: '3'

services:
  develancacheui_backend:
    image: devedse/develancacheui_backend:latest
    restart: unless-stopped
    ports:
      - '7301:80'
    environment:
      - LanCacheLogsDirectory=/var/develancacheui/lancachelogs
      - DeveLanCacheUIDataDirectory=/var/develancacheuidata
      - ConnectionStrings__DefaultConnection=Data Source={DeveLanCacheUIDataDirectory}/database/develancacheui.db;
      - TZ=Europe/Amsterdam
      - LANG=en_GB.UTF-8
    volumes:
      - "/home/pi/dockercomposers/develancacheui/backend:/var/develancacheuidata"
      - "/mnt/mynas/DockerComposers/lancache/logs:/var/develancacheui/lancachelogs:ro"
    dns:
      - 10.88.20.254
  develancacheui_frontend:
    image: devedse/develancacheui_frontend:latest
    restart: unless-stopped
    ports:
      - '7302:80'
    environment:
      - BACKENDURL=http://10.88.20.2:7301 #iclude http/https here

Actual Steam Automatic Update Integration

tpil90 helped me build a feature where the application actually integrates with the Steam API to constantly watch for updated depots. This feature can be activated by adding the following lines to the environment of the backend:

services:
  develancacheui_backend:
    ...
    environment:
      - Feature_DirectSteamIntegration=true
      - Feature_SkipLinesBasedOnBytesRead=true
    ...

This should now all works quite neatly, if it doesn't, let me know in a github issue :smile:.

How it works

DeveLanCacheUI_Backend starts by reading through the whole access.log file generated by LanCache. Besides processing the whole existing access.log file, it will also 'watch' the file for new log lines. All log lines that were already processed before are skipped. It parses every line and tries to create "DownloadEvents". A download event is an event that starts at a specific point in time and ends. Download events are separated/grouped by a unique identifier: clientIp, service, downloadIdentifier (some unique identifier, e.g. DepotId for Steam. Not all services have this) All new log lines that happen within 5 minutes of the last update of the download event with the same unique identifier will be grouped.

Let's say at 00:00 we start downloading CounterStrike on steam. A unique identifier is made: 192.168.2.123,steam,2347771 If a new log entry with the same unique identifier is found, it's searching the db for past event that were updated in the last 5 minutes. If that's the case, this log entry is appended to the existing download event.

This results in download events being specific instances of a certain client download a certain application/update/game.

Issue: My access.log file is updated but my DeveLanCacheUI_Backend isn't reading the new lines

So apparently if the access.log file is in a SMB Share which is mounted in docker (as is the case for me) the DeveLanCacheUI_Backend application takes a READ lock on the share. This apparently lets CIFS decide that no other applications will write to this file which allows it to cache things. If you manually execute the ls command in the lancachelogs directory it will in fact start reading the file again.

To work around this issue you need to add cache=none to the CIFS mount in /etc/fstab:

//192.168.2.201/DockerComposers /mnt/mynas/DockerComposers cifs credentials=/home/pi/.mynascredentialssmb,iocharset=utf8,vers=3.0,sec=ntlmssp,cache=none 0 0

References: https://stackoverflow.com/questions/14920591/how-to-disable-caching-in-cifs-samba-on-client-side-in-linux https://www.samba.org/samba/docs/old/Samba3-HOWTO/locking.html https://linux.die.net/man/8/mount.cifs

Progress / Plans

TODO:

1. Implement auto reload SteamAPI
2. Implement automatically downloading Depot App Mapping .csv file

Build status

GitHubActions Builds

DockerHub

Docker Hub

Code Coverage Status

CodeCov

Code Quality Status

SonarQube