bug report: failed to update data from leetcode.cn

[yao-weijie]

env: leetcodec-cli: 0.3.11 [sys.urls]: changed leetcode.com -> leetcode.cn [cookies]: properly set, can test and submit solutions

executing leetcode data --update, got output:

[INFO  leetcode_cli::cache] Fetching leetcode problems...
json from response parse failed, please open a new issue at: https://github.com/clearloop/
leetcode-cli/.

then change back to leetcode.com, database updated sucessfully

[Fr4nk1inCs]

The cause is that leetcode.com and leetcode.cn respond in different json formats when featching problems.

In leetcode.com 's json response, "stat_status_pairs > stat > frontend_question_id" is a number:

{
    ...
    "stat_status_pairs": [
        {
            "stat": {
                "question_id": 2714,
                "question__article__live": null,
                "question__article__slug": null,
                "question__article__has_video_solution": null,
                "question__title": "Left and Right Sum Differences",
                "question__title_slug": "left-and-right-sum-differences",
                "question__hide": false,
                "total_acs": 28927,
                "total_submitted": 32585,
                "frontend_question_id": 2574, // <- Frontend ID here
                "is_new_question": false
            },
            "status": null,
            "difficulty": {
                "level": 1
            },
            "paid_only": false,
            "is_favor": false,
            "frequency": 0,
            "progress": 0
        },
        ...
    ]
}

However, that field in leetcode.cn's response is a string, whose last word is a number:

{
    ...
    "stat_status_pairs": [
        {
            "stat": {
                "question_id": 1000486,
                "question__title": "\u96c6\u6c34\u5668",
                "question__title_slug": "kskhHQ",
                "question__hide": false,
                "total_acs": 423,
                "total_submitted": 854,
                "total_column_articles": 10,
                "frontend_question_id": "LCP 71", // <- Frontend ID here
                "is_new_question": false
            },
            "status": null,
            "difficulty": {
                "level": 3
            },
            "paid_only": false,
            "is_favor": false,
            "frequency": 0,
            "progress": 0
        },
        ...
    ],
}

A possible temporary workaround for this is that modify the 15th line of src/cache/parser.rs:

/// problem parser
pub fn problem(problems: &mut Vec<Problem>, v: Value) -> Option<()> {
    let pairs = v.get("stat_status_pairs")?.as_array()?;
    for p in pairs {
        let stat = p.get("stat")?.as_object()?;
        let total_acs = stat.get("total_acs")?.as_f64()? as f32;
        let total_submitted = stat.get("total_submitted")?.as_f64()? as f32;

        problems.push(Problem {
            category: v.get("category_slug")?.as_str()?.to_string(),
            // Before:
            // fid: stat.get("frontend_question_id")?.as_i64()? as i32,
            // After:
            fid: stat
                .get("frontend_question_id")?
                .as_str()?
                .split(" ")
                .last()?
                .parse::<i32>()
                .ok()?,
            // End
            id: stat.get("question_id")?.as_i64()? as i32,
            level: p.get("difficulty")?.as_object()?.get("level")?.as_i64()? as i32,
            locked: p.get("paid_only")?.as_bool()?,
            name: stat.get("question__title")?.as_str()?.to_string(),
            percent: total_acs / total_submitted * 100.0,
            slug: stat.get("question__title_slug")?.as_str()?.to_string(),
            starred: p.get("is_favor")?.as_bool()?,
            status: p.get("status")?.as_str().unwrap_or("Null").to_string(),
            desc: String::new(),
        });
    }

    Some(())
}

[wendajiang]

The question is , what is mean about leetcodecn's frontend_question_id, it's same with leetcode.com ?

[Fr4nk1inCs]

Just found that some of the leetcodecn's problem just have the title LCP xx: xxx 😅.

Also found that the daily problem API is also different, the code below fits for CN.

src/plugins/leetcode.rs - get_question_daily()

pub async fn get_question_daily(self) -> Result<Response, Error> {
    trace!("Requesting daily problem...");
    let url = &self.conf.sys.urls.get("graphql").ok_or(Error::NoneError)?;
    let mut json: Json = HashMap::new();
    json.insert("operationName", "questionOfToday".to_string());
    json.insert(
        "query",
        vec![
            "query questionOfToday {",
            "  todayRecord {",
            "    question {",
            "      questionFrontendId",
            "    }",
            "  }",
            "}",
        ]
        .join("\n"),
    );

    Req {
        default_headers: self.default_headers,
        refer: None,
        info: false,
        json: Some(json),
        mode: Mode::Post,
        name: "get_question_daily",
        url: (*url).to_string(),
    }
    .send(&self.client)
    .await
}

src/cache/parser.rs - daily()

/// daily parser
pub fn daily(v: Value) -> Option<i32> {
    trace!("Parse daily...");
    v.as_object()?
        .get("data")?
        .as_object()?
        .get("todayRecord")?
        .as_array()?[0]
        .as_object()?
        .get("question")?
        .as_object()?
        .get("questionFrontendId")?
        .as_str()?
        .parse()
        .ok()
}

[Fr4nk1inCs]

The question is , what is mean about leetcodecn's frontend_question_id, it's same with leetcode.com ?

Pretty sure that most of the problems has the same FID with different type ("1234" <-> 1234), except "LCP" problems.