How to use 2PL & GRM on different items in same model?

[PatrickJEdwards]

Hi Robert Kubinec,

My apologies for all the questions. I'm very excited to use your idealstan package, though I'm having some issues and the package's current documentation doesn't offer a solution. Note again that I'm using the development version of your package.

I'm trying to run another dynamic ideal point model with random-walk priors on a dataset with both binary and ordinal items. I've used two modeling configurations so far, though both didn't run correctly through the package.

Here is example data that you can use to replicate the issues: idealstan_GithubExampleData_FIN_subset.csv

Attempt 1: using both 2PL and GRM

First, I tried to assign different models to different items:

• Binary items: IRT 2-PL (binary response) ideal point model + no missing-data inflation (model_type == 1).
• Ordinal items: ordinal IRT (graded response ) ideal point model + no missing-data inflation (model_type == 5).

I passed this model information to idealstan through the id_make function's model_id argument. Here's the model_id and id_estimate functions I ran to get these results:

idealstan_object <- id_make(
  score_data = idealstan_GithubExampleData_FIN_subset,
  person_id = "person_id",
  item_id = "item_id",
  time_id = "time_id",
  group_id = "group_id",
  model_id = "model_id_2PL_GRM",
  unbounded = FALSE,
  outcome_disc = "outcome_disc",
  ordered_id = "ordered_id"
)

#Specify `idealstan` model with random-walk prior and PRE-FIXED CONSTANTS:
idealstan_output_PREFIX <- id_estimate(
  idealdata = idealstan_object,
  vary_ideal_pts = "random_walk",
  fixtype = "prefix",
  const_type = "items",
  restrict_ind_high = "104",
  restrict_ind_low = "108",
  ncores = parallel::detectCores(),
  grainsize = 1,
  nchains = 4,
  id_refresh = 10,
  time_var = 1, #Default=10. Lower values = LESS conservative prior.
  restrict_var = F
)

#Estimate item parameters:
items_summary <- summary(idealstan_output_PREFIX, pars = "items")

Unfortunately, this model produced the following error:

Warning: Chain 4 finished unexpectedly!
Warning: All chains finished unexpectedly! Use the $output(chain_id) method for more information.
Warning: Use read_cmdstan_csv() to read the results of the failed chains.
Warning: No chains finished successfully. Unable to retrieve the fit.
Error: No chains finished successfully. Unable to retrieve the draws.

I could not produce a different outcome if I tried to use more than one model.

Attempt 2: using GRM only.

Second, I tried to estimate all items---both binary and ordinal items---using the same IRT model: the graded response model (model_type == 5). I did this because, for binary items, the graded response model is mathematically equivalent to the 2PL model.

The model ran without issue. However, when I tried to estimate the item's IRT parameters, I got the following error: Error: Can't find the following variable(s) in the output: steps_votes_grm2[2,1]. I don't think your package currently allows binary items to be estimated using GRM.

Here's the code I used to run that model:

idealstan_object <- id_make(
  score_data = idealstan_GithubExampleData_FIN_subset,
  person_id = "person_id",
  item_id = "item_id",
  time_id = "time_id",
  group_id = "group_id",
  model_id = "model_id_GRM",
  unbounded = FALSE,
  outcome_disc = "outcome_disc",
  ordered_id = "ordered_id"
)

#Specify idealstan model with random-walk prior and PRE-FIXED CONSTANTS:
idealstan_output_PREFIX <- id_estimate(
  idealdata = idealstan_object,
  model_type = 5,
  vary_ideal_pts = "random_walk",
  fixtype = "prefix",
  const_type = "items",
  restrict_ind_high = "104",
  restrict_ind_low = "108",
  ncores = parallel::detectCores(),
  grainsize = 1,
  nchains = 4,
  id_refresh = 10,
  time_var = 1, #Default=10. Lower values = LESS conservative prior.
  restrict_var = F
)

#Estimate item parameters:
items_summary <- summary(idealstan_output_PREFIX, pars = "items")

Again, this produced the error:

Error: Can't find the following variable(s) in the output: steps_votes_grm2[2,1]

[saudiwin]

ok @PatrickJEdwards thanks for the sample data, I will take a look this week and get back to you. These models work fine but there may have been an upstream bug introduced somewhere.