N_m3u8DL-RE icon indicating copy to clipboard operation
N_m3u8DL-RE copied to clipboard

多进程调用Nm3u8DL-RE时,如何才能规则排列多个窗口的进度

Open xueli12 opened this issue 1 year ago • 7 comments

我在python中采用多进程的方式同时下载多个hls视频,但是无法解决的问题是窗口的进度会比较混乱。该如何解决这个问题呀 E2 $%%9$PE0N{)%@ 6QRI43

xueli12 avatar Feb 02 '24 09:02 xueli12

每个子线程再读取时候解析进度条信息,然后写入一个dict 主线程每隔几秒输出一下dict内容

RikaCelery avatar Feb 02 '24 13:02 RikaCelery

每个子线程再读取时候解析进度条信息,然后写入一个dict 主线程每隔几秒输出一下dict内容

感谢老哥的建议

不过我解析不出进度条。我让GPT写的代码,不知道哪里出问题了:

`def download_process(download_cmd, config, cookie, m3u8_link, video_id): global last_vid_lines # 声明全局变量 last_vid_line = None with subprocess.Popen(download_cmd, stdout=subprocess.PIPE, bufsize=1, universal_newlines=True, encoding='utf-8') as p: for raw_line in p.stdout: try: line = raw_line.decode('utf-8') # 尝试utf-8解码 except UnicodeDecodeError: # 如果utf-8解码失败 try: line = raw_line.decode('gbk') # 尝试gbk解码 except UnicodeDecodeError: line = raw_line.decode('latin-1', errors='replace') # 尝试latin-1解码,如果还是失败,就替换错误的字符

        if "Vid" in line:
            last_vid_line = line  # 只保留包含Vid的最后的一行
            last_vid_lines[video_id] = last_vid_line  # 更新全局变量中的下载进度

        if "One or more errors occurred" in line:
            thread_progress[video_id] = 'retrying'
            print(f'{video_id}: One or more errors occurred. Retrying...')
            download_video(config, cookie, m3u8_link, video_id)
    thread_progress[video_id] = 'finished'
    if last_vid_line is not None:
        print(f'Latest Vid Progress for {video_id}: {last_vid_line}')  # 打印最后更新的进度条信息

def print_progress(): while True: time.sleep(1) # 每秒输出一次 for video_id, progress in last_vid_lines.items(): print(f'Latest Vid Progress for {video_id}: {progress}')`

xueli12 avatar Feb 02 '24 15:02 xueli12

看的有点累人,请这样上传代码 image 先休息了,明天抽空看

RikaCelery avatar Feb 02 '24 16:02 RikaCelery

看的有点累人,请这样上传代码 image 先休息了,明天抽空看

# 使用Popen实时获取子进程输出
def download_process(download_cmd, config, cookie, m3u8_link, video_id):
    global last_vid_lines  # 声明全局变量
    last_vid_line = None
    with subprocess.Popen(download_cmd, stdout=subprocess.PIPE, bufsize=1, universal_newlines=True, encoding='utf-8') as p:
        for raw_line in p.stdout:
            try:
                line = raw_line.decode('utf-8')  # 尝试utf-8解码
            except UnicodeDecodeError:  # 如果utf-8解码失败
                try:
                    line = raw_line.decode('gbk')  # 尝试gbk解码
                except UnicodeDecodeError:
                    line = raw_line.decode('latin-1', errors='replace')  # 尝试latin-1解码,如果还是失败,就替换错误的字符

            if "Vid" in line:
                last_vid_line = line  # 只保留包含Vid的最后的一行
                last_vid_lines[video_id] = last_vid_line  # 更新全局变量中的下载进度

            if "One or more errors occurred" in line:
                thread_progress[video_id] = 'retrying'
                print(f'{video_id}: One or more errors occurred. Retrying...')
                download_video(config, cookie, m3u8_link, video_id)
        thread_progress[video_id] = 'finished'
        if last_vid_line is not None:
            print(f'Latest Vid Progress for {video_id}: {last_vid_line}')  # 打印最后更新的进度条信息

# 新增的定时器线程,用于定期输出下载进度
def print_progress():
    while True:
        time.sleep(1)  # 每秒输出一次
        for video_id, progress in last_vid_lines.items():
            print(f'Latest Vid Progress for {video_id}: {progress}')

xueli12 avatar Feb 02 '24 16:02 xueli12

看的有点累人,请这样上传代码 image 先休息了,明天抽空看

# 使用Popen实时获取子进程输出
def download_process(download_cmd, config, cookie, m3u8_link, video_id):
    global last_vid_lines  # 声明全局变量
    last_vid_line = None
    with subprocess.Popen(download_cmd, stdout=subprocess.PIPE, bufsize=1, universal_newlines=True, encoding='utf-8') as p:
        for raw_line in p.stdout:
            try:
                line = raw_line.decode('utf-8')  # 尝试utf-8解码
            except UnicodeDecodeError:  # 如果utf-8解码失败
                try:
                    line = raw_line.decode('gbk')  # 尝试gbk解码
                except UnicodeDecodeError:
                    line = raw_line.decode('latin-1', errors='replace')  # 尝试latin-1解码,如果还是失败,就替换错误的字符

            if "Vid" in line:
                last_vid_line = line  # 只保留包含Vid的最后的一行
                last_vid_lines[video_id] = last_vid_line  # 更新全局变量中的下载进度

            if "One or more errors occurred" in line:
                thread_progress[video_id] = 'retrying'
                print(f'{video_id}: One or more errors occurred. Retrying...')
                download_video(config, cookie, m3u8_link, video_id)
        thread_progress[video_id] = 'finished'
        if last_vid_line is not None:
            print(f'Latest Vid Progress for {video_id}: {last_vid_line}')  # 打印最后更新的进度条信息

# 新增的定时器线程,用于定期输出下载进度
def print_progress():
    while True:
        time.sleep(1)  # 每秒输出一次
        for video_id, progress in last_vid_lines.items():
            print(f'Latest Vid Progress for {video_id}: {progress}')

哈哈你这样写肯定是不行的,如果不介意的话,你可以发一下完整代码,把关键信息隐藏一下,我改完给你发回去

RikaCelery avatar Feb 03 '24 00:02 RikaCelery

我之前写过一个闹着玩的东西,使用正则表达式提取的进度信息 image


                //Vid 286x160 | 414 Kbps | 30 12m34s/12m34s 376/378 Recording  99% 0.00Bps(2)
                // 处理每一行的输出
                if (e.Data != null && !string.IsNullOrWhiteSpace(e.Data))
                {
                    Console.WriteLine("输出行: " + e.Data);
                    matchContentType(e.Data);
                    TaskLogs = e.Data;
                    //(\d+)x(\d+) \| (\d+) Kbps \| (\d+) (?:(\d+)h)?(\d+)m(\d+)s\/(?:(\d+)h)?(\d+)m(\d+)s 
                    // (Recording|Waiting)
                    Regex regex =
                        new Regex(
                            @"(\d+)\/(\d+)");
                    Match match = regex.Match(e.Data);

                    if (match.Success)
                    {
                        if (e.Data.IndexOf("Recording") > 0)
                        {
                            TaskState = State.Running;
                        }
                        else if (e.Data.IndexOf("Waiting") > 0)
                        {
                            TaskState = State.Init;
                        }

                        var regex1 = new Regex(@"(?:(-|\d+\.\d+)([KM]?Bps)?)(?:\((\d+)\))?");

                        TaskState = State.Running;

                        var match1 = regex1.Match(e.Data);
                        string speed = match1.Groups[1].Value;
                        long byteSpeed = 0;
                        if (speed == "-")
                        {
                            speed = "0";
                        }

                        double value = double.Parse(speed);
                        switch (match1.Groups[2].Value)
                        {
                            case "MBps":
                                byteSpeed = (long)(value * 1024 * 1024);
                                break;
                            case "KBps":
                                byteSpeed = (long)(value * 1024);
                                break;
                            case "Bps":
                                byteSpeed = (long)value;
                                break;
                        }

                        var detail = () =>
                        {
                            if (ProgressDetail != null && ProgressDetail.Split('/').Length == 2)
                            {
                                var select = ProgressDetail.Split('/').Select(int.Parse).ToArray();
                                return (int)(select[0] * 100.0 / select[1]);
                            }
                            else
                            {
                                return 0;
                            }
                        };
                        var color = () =>
                        {
                            if (ProgressDetail != null && ProgressDetail.Split('/').Length == 2)
                            {
                                var select = ProgressDetail.Split('/').Select(int.Parse).ToArray();
                                var mapValueToColor = ColorMapper.MapValueToColor(TaskProgress);
                                return mapValueToColor.ToBrush();
                            }
                            else
                            {
                                return Colors.Black.ToBrush();
                            }
                        };
                        App.GetService<DashBoardPage>()!.Dispatcher.Invoke(() =>
                        {

                            ProgressDetail = match.Groups[1].Value + "/" + match.Groups[2].Value;
                            Retrys = match1.Groups[3].Value;
                            TaskSpeed = byteSpeed;
                            TaskProgress = detail();
                            Color = color();
                        });
                    }

                    var logRegex = new Regex(@"\d+:\d+:\d+\.\d+ (\S+) :");
                    var logMatch = logRegex.Match(e.Data);
                    if (logMatch.Success)
                    {
                        switch (logMatch.Groups[1].Value)
                        {
                            case "DEBUG": break;
                            case "INFO": break;
                            case "WARN": break;
                            case "ERROR":
                                TaskState = State.Error;
                                break;
                        }
                    }
                }

RikaCelery avatar Feb 03 '24 01:02 RikaCelery

感谢大佬,是个很棒的思路,我再去尝试一下

xueli12 avatar Feb 03 '24 10:02 xueli12