Linux Support?

[nwvh]

Hey, would it be possible to make Linux support as it's a python file? Thanks

[DevilXD]

Yes, the thing is, the GUI suite I'm using should work properly on Linux as well, and all cases where a Windows-only code is used, already have some extra conditions to avoid throwing an error. This code just never been tested on a Linux - you can try running it there from source, and report back any errors or issues that'd appear.

If everything would work, I can think about adding a release packaging flow for Linux as well.

[nwvh]

So I tried it and I have a problem with pywin32 library, which is only available for windows, I tried workarounds but was unsuccessful...

[DevilXD]

Traceback please. I know this library won't work on Linux, but all places where it's used, already have a condition in place that makes it work without the library. At least in theory - in practice, I'd need the traceback to see where the issue is.

[bricep]

[bricep]

And when attempting to not import pywin32 within main.py:

[DevilXD]

@bricep I see, thank you for taking interest! Because the Linux support was never really planned when I was making this application, we can both expect some heavy steps (and hopefully not impassable walls) along the way, hence why I've created a new branch for testing this: https://github.com/DevilXD/TwitchDropsMiner/tree/linux

Please give it a try and let me know how far it gets you this time.

Also, just a cursory question - are you trying to run this on a headless machine? Because this application comes with a GUI that currently cannot handle headless in any capacity, mostly due to the non-headless Chrome login flow that's required for authentication.

[bricep]

I completely understand and if your time is limited, don't feel obligated or that you have to troubleshoot or work on this at all. I was just happy to send a traceback and not leave you hanging!

I did a fresh pull from the new linux branch and get to the same place as before:

To answer your question, this isn't a headless machine.

[DevilXD]

Eh, I thought this one was protected already. How is it now?

[bricep]

First Then after installing seleniumwire, I get this error and have to install undetected_chromium: I got an error about importing PIL and ended up having to upgrade to the latest version. Then, I got this:

[DevilXD]

All required libraries can be installed via pip install -r requirements.txt, and they normally do so if you'd run the setup_env.bat file. Although, Linux cannot run it, so there's that. The winreg module is used for autostart handling, not sure how I'd be able to achieve this on Linux yet.

It took me a while to figure out how to handle this import, as I only intend to proceed with the loading and disable the functionality wherever needed. Proper care would need to be taken later on when actually handling all of these cases, for now we're trying to get this thing to load at all. Try again please?

[bricep]

Awesome! The requirements error out at pywin32... pip won't install it. After that, I get this error now when I run main.py:

I have confirmed the icon file is present and isn't corrupted. I found this old thread and attempted to use an xbm version but that threw the same error.

[DevilXD]

I could split the requirements into windows and linux ones easily, that should solve that.

The probable cause is that the ICO file format is windows-specific. I've ran out of time today, but I'll get back to this tomorrow and over the weekend, see what I can do.

[sanjuant]

Hi, I managed to launch the program under Linux by modifying the lines that refer to the icon.

First of all, I converted the icon to GIF.

# handle input parameters
    # NOTE: parser output is shown via message box
    # we also need a dummy invisible window for the parser
    root = tk.Tk()
    root.overrideredirect(True)
    root.withdraw()
    icon = tk.PhotoImage(file='pickaxe.gif')
    root.iconphoto(True, icon)
    root.update()

            self.icon = pystray.Icon(
                "twitch_miner",
                Image_module.open(resource_path("pickaxe.gif")),
                self.get_title(drop),
                menu,
            )

        # withdraw immediately to prevent the window from flashing
        self._root.withdraw()
        # root.resizable(False, True)
        icon = tk.PhotoImage(file='pickaxe.gif')
        root.iconphoto(True, icon)
        root.title(WINDOW_TITLE)  # window title
        root.bind_all("<KeyPress-Escape>", self.unfocus)  # pressing ESC unfocuses selection

It's possible to run in headless mode to dockerize app ?

[DevilXD]

It's possible to run in headless mode to dockerize app ?

This belongs more to #17 rather than here, but the short answer is no, because the miner is currently strictly tied with the GUI framework, which you can't control from headless. And even if I would want to make it headless, then I can't because of the non-headless Chrome backup login flow. If you'd still want to say something about this topic, please do so under #17.

Regarding the rest, yeah, the icon format was being the culprit here, and I just had not enough free time to really get to it. I was also wondering which file format I could use for this, that'd support and make sense on both windows and linux. Your GIF solution may seem like it's working, but not really (pic from windows for comparison):

It could be just a result of local vars being garbage-collected, and the icon variable's contents ceasing from existence - had that happen to me before. Just a matter of keeping a reference to the photo image somewhere. I was gonna use PNG as a more system-independent format, but it looks like the GIF is the only option: https://stackoverflow.com/a/27980230/8576445

Even with all of this being "solved", there's still a matter of the functionality that had to be "disabled" to make this thing run on Linux, specifically:

• Chrome backup login flow possibly starting with an extra console window (could be possibly remedied with some documentation)
• Autostart support (no windows registry here, not sure how that's even done on linux)
• No duplicated instance protection
• Questionable support for graceful exit during system shutdown (there's code for this already, but it was never tested)

Not sure what to do with these cases.

@sanjuant Have you done any additional usage tests (using the gui, logging in and trying to mine) besides launching it?

[sanjuant]

Thanks for your answer, I think the icon didn't show up because I'm running it under WSL2.

The browser doesn't seem to be compatible, and as I wanted a docker solution, I didn't investigate further. Can I insert a cookie somewhere to bypass the login part?

[DevilXD]

The backup Chrome login currently ceased to work due to Kasada (Twitch's anti-bot protection) updating against the undetected_chromedriver. I've opened an issue regarding that here: https://github.com/ultrafunkamsterdam/undetected-chromedriver/issues/897

There's a different, old way of logging in available in the latest master, which I've just merged into the linux branch. Please try using that instead. If the chrome browser would open again, see which login error code you've got, it should be printed out in the output window.

[sanjuant]

I forgot to mention that I had to do without an import to run the program.

❯ python main.py
Traceback (most recent call last):
  File "main.py", line 41, in <module>
    from twitch import Twitch
  File "/home/sorrow/TwitchDropsMiner/twitch.py", line 12, in <module>
    from subprocess import CREATE_NO_WINDOW
ImportError: cannot import name 'CREATE_NO_WINDOW' from 'subprocess' (/usr/lib/python3.8/subprocess.py)
❯ python main.py
Traceback (most recent call last):
  File "main.py", line 108, in <module>
    root.iconbitmap(resource_path("pickaxe.ico"))
  File "/usr/lib/python3.8/tkinter/__init__.py", line 2080, in wm_iconbitmap
    return self.tk.call('wm', 'iconbitmap', self._w, bitmap)
_tkinter.TclError: bitmap "/home/sorrow/TwitchDropsMiner/pickaxe.ico" not defined
❯ python main.py

As you can see the icon works in the taskbar.

After that I can connect, the points seem to recover well but it seems that this is the only thing that works. I clicked on "Load Points" but nothing happened.

In inventory, Refresh not work.

In settings, i can add or exclude games.

Reload button seem to trigger campains.

The language works but I think I don't have the Chinese characters installed on my linux system.

Some Twitch campaigns listed on my profile.

[DevilXD]

ImportError: cannot import name 'CREATE_NO_WINDOW' from 'subprocess'

That's my fault, it was already handled previously (https://github.com/DevilXD/TwitchDropsMiner/commit/3fdeb7354b75f7f911c2c76c09e235288c9cdbe9), but I messed up the merge commit (https://github.com/DevilXD/TwitchDropsMiner/commit/b86dd3cf804bebd6a95364608943ae634cd70435) and it got overwritten by the master branch. I've fixed it yet again now: https://github.com/DevilXD/TwitchDropsMiner/commit/c80bccdfa6537fb8954b5a2740e9d5b9f63e8b13

_tkinter.TclError: bitmap "/home/sorrow/TwitchDropsMiner/pickaxe.ico" not defined

Pretty sure that one is from one of the icon changes you made. I just pushed out the icon change of my own, see if it works for you with no further changes required or not please. Additional changes included gating pywin32 behind a windows check, so that requirements.txt can be used for library installation with no modifications.

"Load points" won't do anything if there's no channels on the channel list. On the inventory tab, to see all campaigns, you have to check all filters, uncheck "Linked only", and only then press the "Refresh" button. To mine a campaign, ensure it's state is "Active" and it shows your account is "Linked" - if it's not, click on the "Not linked" text to proceed to linking. Links can also be handled from the campaigns page: https://www.twitch.tv/drops/campaigns With a linked campaign, ensure it's on the Priority list, or uncheck the "Priority only" option, then hit "Reload" to see if everything works.

I haven't considered the Chinese characters not being visible on some systems. I might change the language names to be strictly English names (Chinese, German, Chech) to avoid this.

[sanjuant]

_tkinter.TclError: bitmap "/home/sorrow/TwitchDropsMiner/pickaxe.ico" not defined

Pretty sure that one is from one of the icon changes you made. I just pushed out the icon change of my own, see if it works for you with no further changes required or not please. Additional changes included gating pywin32 behind a windows check, so that requirements.txt can be used for library installation with no modifications.

This error occurs before I have changed anything on my end. With new version, icon work.

"Load points" won't do anything if there's no channels on the channel list. On the inventory tab, to see all campaigns, you have to check all filters, uncheck "Linked only", and only then press the "Refresh" button. To mine a campaign, ensure it's state is "Active" and it shows your account is "Linked" - if it's not, click on the "Not linked" text to proceed to linking. Links can also be handled from the campaigns page: https://www.twitch.tv/drops/campaigns With a linked campaign, ensure it's on the Priority list, or uncheck the "Priority only" option, then hit "Reload" to see if everything works.

If in channel you only see the current campaigns then everything is ok :) I thought I would see the channels I follow that are online there.

The only error I could see is the characters that are not displayed.

[DevilXD]

Glad it works =) Can replace the Chinese characters, but there's no good replacement for emojis. In the last pic, I could theoretically just use a capital X, but that's an eh solution.

I think it's safe to merge the linux branch into the master branch now.

[WrayOfSunshine]

Can someone write a condensed "how-to" and add it to the wiki? I'm trying to follow along in this issue's comments but it'd be much easier if we had a page that directly explained how to run it.

[DevilXD]

@dethmourne I just did so today though, here's a link: https://github.com/DevilXD/TwitchDropsMiner/wiki/Setting-up-the-environment,-building-and-running

It's as condensed as it can get, really couldn't make it simpler. Can't auto-install Python and Git with proper configuration, but at least once you do so, it's all just a matter of running some batch scripts that do all the thinking and calls for you. The only thing that's still missing is maintenance section AKA what to do when a new version of source code becomes available, but I'll add it soon / at some point in the future.

That's for running this project from the Github's master branch, which always has the latest changes implemented. Normally, the newest stable version is available in releases: https://github.com/DevilXD/TwitchDropsMiner/releases, although the current v14 still uses a broken login system, and I'm still fixing things for the v15's release.

[WrayOfSunshine]

@DevilXD To the best of my knowledge you can't run .bat files in linux, though.

[WrayOfSunshine]

Okay, apparently I was wrong on that front. I do get output from the build.bat but it says:

./build.bat: line 1: @echo: command not found ./build.bat: line 2: cls: command not found ./build.bat: line 9: unexpected EOF while looking for matching `"' ./build.bat: line 10: syntax error: unexpected end of file

[WrayOfSunshine]

I was able to get it running by just running pip install -r requirements.txt and then running python main.py and it works :)

[DevilXD]

you can't run .bat files in linux, though.

Right. Sorry, I'm not a linux user >.>

I should make a set of .sh files for linux then. Never did so, so idk how hard it'd be.

In terms of installation, what you did was install all required libraries and run it from your global Python installation. This is barely okay for a new Python installation, and only gets worse from there. If you'd ever consider using another project and install it's libraries too, PyInstaller would bundle them when building an executable for this project. I've ran into this at v7 where a built executable was 48 MB (vs current 11 MB) big and took 10+ seconds (1-2 seconds now) to open on my machine. For this reason, there's all of this virtual environment (venv for short) creation process in the env folder and running it from there. I should add some manual instructions to that page too, it seems.

There's no good way for you to undo it for your global installation, but you can still go through the venv process to separate the libraries. A venv is like a separate space for libraries to be installed into, with it's own separate python, pythonw and pip executables. In order to create one, install libraries to it and run the application from there:

• From the project's directory as cwd: python -m venv env, to create a venv in the env folder.
• ./env/Scripts/activate.sh to activate the venv. This should add an (env) ... prefix to your command prompt.
• pip install -r requirements.txt to install required libraries into the venv.
• pythonw main.py to run the application.

Instead of the venv activation step, you can use ./env/Scripts/python, ./env/Scripts/pythonw and ./env/Scripts/pip executables directly, to install the requirements and later on, run the application.

[sanjuant]

Hi @DevilXD ,

This is command i have typed to execute on WSL2

pip install --user virtualenv
git pull
virtualenv -p python3 venv
source venv/bin/activate
pip install -r requirements.txt
python3 main.py

I don't think pythonw works on Linux. But I may be wrong.

[DevilXD]

@sanjuant You don't need to pip install a 3rd party library for this, Python 3 comes with a venv module in it's standard library: https://docs.python.org/3/library/venv.html

All you need to do is run python -m venv <path> to create a new venv in the specified <path>. Rest of the commands looks okay.

[Juice805]

Been using https://github.com/Zaarrg/TTVDropBot for some time now, but it has fallen into disarray. A lightweight docker image release (dependent on #17 and linux support) would instantly take its place on my server.

I'll be messing around with it for fun in the meantime

[DevilXD]

@Juice805 The headless mode is still on the table, but not the docker image - you'd have to create that one yourself. Also please note that this project is more or less a personal miner rather than something you're supposed to run 24/7 on some kind of server.

I'm still wondering how I could tackle the headless mode, as all effort put into the project since v4 was to try and convert this thing from an "ugly" console application, into something that has a nice user interface. UI code is now strictly tied to virtually every step the miner makes, and there's no easy way of "decoupling" it from the code, short of writing a separate client with no UI interaction at all.