pandas
pandas copied to clipboard
pandas-dev
BUG: Unable to open Stata 118 or 119 format files saved in big-endian…
… format that contain strL data
- [x] closes #58638
- [x] Tests added and passed if fixing a bug or adding a new feature
- [x] All code checks passed.
- [ ] Added type annotations to new arguments/methods/functions.
- [x] Added an entry in the latest
doc/source/whatsnew/vX.X.X.rstfile if fixing a bug or adding a new feature.
These files were not created with Stata but were made using the published specification. I have however checked that they all load correctly into Stata and have used it to verify that they are constructed correctly:
. use stata12_118.dta
. list
+------------------------------+
| x y z |
|------------------------------|
1. | 1 abc abcdefghi |
2. | 3 cba qwertywertyqwerty |
3. | 93 strl |
+------------------------------+
. dtaverify stata12_118.dta
(file "stata12_118.dta" is .dta-format 118 from Stata 14, 15, 16, 17, or 18)
1. reading and verifying header
release is 118
byteorder LSF
K (# of vars) is 3
N (# of obs) is 3
label length 0 ||
date length 17 |19 Feb 2015 09:20|
2. reading and verifying map
map[ 1] = 0
map[ 2] = 158
map[ 3] = 281
map[ 4] = 320
map[ 5] = 728
map[ 6] = 757
map[ 7] = 947
map[ 8] = 1373
map[ 9] = 2371
map[10] = 2406
map[11] = 2473
map[12] = 2581
map[13] = 2610
map[14] = 2622
verifying map
1 2 3 4 5 6 7 8 9 10 11 12 13 14
3. reading and verifying vartypes
4. reading and verifying varnames
verifying varnames unique
5. reading and verifying sort order
verifying contents
6. reading and verifying display formats
verifying formats
verifying formats correspond to variable type
7. reading and verifying value-label assignment
verifying value-label construction
verifying value label and corresponding variable types
8. reading and verifying variable-label assignment
verifying variable labels construction
9. reading and verifying characteristics
(0 characteristics in file)
10. reading and verifying data
(dtaverify_118 cannot verify that values are correct)
verifying construction
(1 strL variables)
verifying strL construction
11. reading and verifying strLs
verifying construction
(3 strLs expected)
12. reading and verifying value label definitions
verifying construction
(0 labels in file)
no errors or warnings above; .dta file valid
. use stata12_119.dta
. list
+------------------------------+
| x y z |
|------------------------------|
1. | 1 abc abcdefghi |
2. | 3 cba qwertywertyqwerty |
3. | 93 strl |
+------------------------------+
. dtaverify stata12_119.dta
(file "stata12_119.dta" is .dta-format 119 from Stata 15, 16, 17, or 18)
1. reading and verifying header
release is 119
byteorder LSF
K (# of vars) is 3
N (# of obs) is 3
label length 0 ||
date length 17 |19 Feb 2015 09:20|
2. reading and verifying map
map[ 1] = 0
map[ 2] = 160
map[ 3] = 283
map[ 4] = 322
map[ 5] = 730
map[ 6] = 767
map[ 7] = 957
map[ 8] = 1383
map[ 9] = 2381
map[10] = 2416
map[11] = 2483
map[12] = 2591
map[13] = 2620
map[14] = 2632
verifying map
1 2 3 4 5 6 7 8 9 10 11 12 13 14
3. reading and verifying vartypes
4. reading and verifying varnames
verifying varnames unique
5. reading and verifying sort order
verifying contents
6. reading and verifying display formats
verifying formats
verifying formats correspond to variable type
7. reading and verifying value-label assignment
verifying value-label construction
verifying value label and corresponding variable types
8. reading and verifying variable-label assignment
verifying variable labels construction
9. reading and verifying characteristics
(0 characteristics in file)
10. reading and verifying data
(dtaverify_119 cannot verify that values are correct)
verifying construction
(1 strL variables)
verifying strL construction
11. reading and verifying strLs
verifying construction
(3 strLs expected)
12. reading and verifying value label definitions
verifying construction
(0 labels in file)
no errors or warnings above; .dta file valid
. use stata12_be_117.dta
. list
+------------------------------+
| x y z |
|------------------------------|
1. | 1 abc abcdefghi |
2. | 3 cba qwertywertyqwerty |
3. | 93 strl |
+------------------------------+
. dtaverify stata12_be_117.dta
(file "stata12_be_117.dta" is .dta-format 117 from Stata 13)
1. reading and verifying header
release is 117
byteorder MSF
K (# of vars) is 3
N (# of obs) is 3
label length 0 ||
date length 17 |19 Feb 2015 09:20|
2. reading and verifying map
map[ 1] = 0
map[ 2] = 153
map[ 3] = 276
map[ 4] = 315
map[ 5] = 435
map[ 6] = 464
map[ 7] = 630
map[ 8] = 768
map[ 9] = 1046
map[10] = 1081
map[11] = 1148
map[12] = 1244
map[13] = 1273
map[14] = 1285
verifying map
1 2 3 4 5 6 7 8 9 10 11 12 13 14
3. reading and verifying vartypes
4. reading and verifying varnames
verifying varnames unique
5. reading and verifying sort order
verifying contents
6. reading and verifying display formats
verifying formats
verifying formats correspond to variable type
7. reading and verifying value-label assignment
verifying value-label construction
verifying value label and corresponding variable types
8. reading and verifying variable-label assignment
verifying variable labels construction
9. reading and verifying characteristics
(0 characteristics in file)
10. reading and verifying data
(dtaverify_117 cannot verify that values are correct)
verifying construction
(1 strL variables)
verifying strL construction
11. reading and verifying strLs
verifying construction
(3 strLs expected)
12. reading and verifying value label definitions
verifying construction
(0 labels in file)
no errors or warnings above; .dta file valid
. use stata12_be_118.dta
. list
+------------------------------+
| x y z |
|------------------------------|
1. | 1 abc abcdefghi |
2. | 3 cba qwertywertyqwerty |
3. | 93 strl |
+------------------------------+
. dtaverify stata12_be_118.dta
(file "stata12_be_118.dta" is .dta-format 118 from Stata 14, 15, 16, 17, or 18)
1. reading and verifying header
release is 118
byteorder MSF
K (# of vars) is 3
N (# of obs) is 3
label length 0 ||
date length 17 |19 Feb 2015 09:20|
2. reading and verifying map
map[ 1] = 0
map[ 2] = 158
map[ 3] = 281
map[ 4] = 320
map[ 5] = 728
map[ 6] = 757
map[ 7] = 947
map[ 8] = 1373
map[ 9] = 2371
map[10] = 2406
map[11] = 2473
map[12] = 2581
map[13] = 2610
map[14] = 2622
verifying map
1 2 3 4 5 6 7 8 9 10 11 12 13 14
3. reading and verifying vartypes
4. reading and verifying varnames
verifying varnames unique
5. reading and verifying sort order
verifying contents
6. reading and verifying display formats
verifying formats
verifying formats correspond to variable type
7. reading and verifying value-label assignment
verifying value-label construction
verifying value label and corresponding variable types
8. reading and verifying variable-label assignment
verifying variable labels construction
9. reading and verifying characteristics
(0 characteristics in file)
10. reading and verifying data
(dtaverify_118 cannot verify that values are correct)
verifying construction
(1 strL variables)
verifying strL construction
11. reading and verifying strLs
verifying construction
(3 strLs expected)
12. reading and verifying value label definitions
verifying construction
(0 labels in file)
no errors or warnings above; .dta file valid
. use stata12_be_119.dta
. list
+------------------------------+
| x y z |
|------------------------------|
1. | 1 abc abcdefghi |
2. | 3 cba qwertywertyqwerty |
3. | 93 strl |
+------------------------------+
. dtaverify stata12_be_119.dta
(file "stata12_be_119.dta" is .dta-format 119 from Stata 15, 16, 17, or 18)
1. reading and verifying header
release is 119
byteorder MSF
K (# of vars) is 3
N (# of obs) is 3
label length 0 ||
date length 17 |19 Feb 2015 09:20|
2. reading and verifying map
map[ 1] = 0
map[ 2] = 160
map[ 3] = 283
map[ 4] = 322
map[ 5] = 730
map[ 6] = 767
map[ 7] = 957
map[ 8] = 1383
map[ 9] = 2381
map[10] = 2416
map[11] = 2483
map[12] = 2591
map[13] = 2620
map[14] = 2632
verifying map
1 2 3 4 5 6 7 8 9 10 11 12 13 14
3. reading and verifying vartypes
4. reading and verifying varnames
verifying varnames unique
5. reading and verifying sort order
verifying contents
6. reading and verifying display formats
verifying formats
verifying formats correspond to variable type
7. reading and verifying value-label assignment
verifying value-label construction
verifying value label and corresponding variable types
8. reading and verifying variable-label assignment
verifying variable labels construction
9. reading and verifying characteristics
(0 characteristics in file)
10. reading and verifying data
(dtaverify_119 cannot verify that values are correct)
verifying construction
(1 strL variables)
verifying strL construction
11. reading and verifying strLs
verifying construction
(3 strLs expected)
12. reading and verifying value label definitions
verifying construction
(0 labels in file)
no errors or warnings above; .dta file valid
. use stata14_119.dta
(This is a Ünicode data label)
. list
+---------------------------------------------------------------------------+
| Things Cities Unicode~l Ints Floats Bytes Longs |
|---------------------------------------------------------------------------|
1. | Cat Bogota Bogotá 1 1 option b Ünicode 1 |
2. | Dog Boston Uzunköprü . . . . |
3. | Plane Rome Tromsø 0 0 option a 0 |
4. | Potato Tokyo Elâzığ -4 4 4 4 |
5. | 0 .3333 option a .3333333 |
+---------------------------------------------------------------------------+
. dtaverify stata14_119.dta
(file "stata14_119.dta" is .dta-format 119 from Stata 15, 16, 17, or 18)
1. reading and verifying header
release is 119
byteorder LSF
K (# of vars) is 7
N (# of obs) is 5
label length 30 |This is a Ünicode data label|
date length 17 |28 Apr 2015 17:58|
2. reading and verifying map
map[ 1] = 0
map[ 2] = 190
map[ 3] = 313
map[ 4] = 360
map[ 5] = 1284
map[ 6] = 1337
map[ 7] = 1755
map[ 8] = 2697
map[ 9] = 4979
map[10] = 5014
map[11] = 5202
map[12] = 5335
map[13] = 5562
map[14] = 5574
verifying map
1 2 3 4 5 6 7 8 9 10 11 12 13 14
3. reading and verifying vartypes
4. reading and verifying varnames
verifying varnames unique
5. reading and verifying sort order
verifying contents
6. reading and verifying display formats
verifying formats
verifying formats correspond to variable type
7. reading and verifying value-label assignment
verifying value-label construction
verifying value label and corresponding variable types
8. reading and verifying variable-label assignment
verifying variable labels construction
9. reading and verifying characteristics
(0 characteristics in file)
10. reading and verifying data
(dtaverify_119 cannot verify that values are correct)
verifying construction
(1 strL variables)
verifying strL construction
11. reading and verifying strLs
verifying construction
(4 strLs expected)
12. reading and verifying value label definitions
verifying construction
(1 labels in file)
no errors or warnings above; .dta file valid
. use stata14_be_118.dta
(This is a Ünicode data label)
. list
+---------------------------------------------------------------------------+
| Things Cities Unicode~l Ints Floats Bytes Longs |
|---------------------------------------------------------------------------|
1. | Cat Bogota Bogotá 1 1 option b Ünicode 1 |
2. | Dog Boston Uzunköprü . . . . |
3. | Plane Rome Tromsø 0 0 option a 0 |
4. | Potato Tokyo Elâzığ -4 4 4 4 |
5. | 0 .3333 option a .3333333 |
+---------------------------------------------------------------------------+
. dtaverify stata14_be_118.dta
(file "stata14_be_118.dta" is .dta-format 118 from Stata 14, 15, 16, 17, or 18)
1. reading and verifying header
release is 118
byteorder MSF
K (# of vars) is 7
N (# of obs) is 5
label length 30 |This is a Ünicode data label|
date length 17 |28 Apr 2015 17:58|
2. reading and verifying map
map[ 1] = 0
map[ 2] = 188
map[ 3] = 311
map[ 4] = 358
map[ 5] = 1282
map[ 6] = 1319
map[ 7] = 1737
map[ 8] = 2679
map[ 9] = 4961
map[10] = 4996
map[11] = 5184
map[12] = 5317
map[13] = 5544
map[14] = 5556
verifying map
1 2 3 4 5 6 7 8 9 10 11 12 13 14
3. reading and verifying vartypes
4. reading and verifying varnames
verifying varnames unique
5. reading and verifying sort order
verifying contents
6. reading and verifying display formats
verifying formats
verifying formats correspond to variable type
7. reading and verifying value-label assignment
verifying value-label construction
verifying value label and corresponding variable types
8. reading and verifying variable-label assignment
verifying variable labels construction
9. reading and verifying characteristics
(0 characteristics in file)
10. reading and verifying data
(dtaverify_118 cannot verify that values are correct)
verifying construction
(1 strL variables)
verifying strL construction
11. reading and verifying strLs
verifying construction
(4 strLs expected)
12. reading and verifying value label definitions
verifying construction
(1 labels in file)
no errors or warnings above; .dta file valid
. use stata14_be_119.dta
(This is a Ünicode data label)
. list
+---------------------------------------------------------------------------+
| Things Cities Unicode~l Ints Floats Bytes Longs |
|---------------------------------------------------------------------------|
1. | Cat Bogota Bogotá 1 1 option b Ünicode 1 |
2. | Dog Boston Uzunköprü . . . . |
3. | Plane Rome Tromsø 0 0 option a 0 |
4. | Potato Tokyo Elâzığ -4 4 4 4 |
5. | 0 .3333 option a .3333333 |
+---------------------------------------------------------------------------+
. dtaverify stata14_be_119.dta
(file "stata14_be_119.dta" is .dta-format 119 from Stata 15, 16, 17, or 18)
1. reading and verifying header
release is 119
byteorder MSF
K (# of vars) is 7
N (# of obs) is 5
label length 30 |This is a Ünicode data label|
date length 17 |28 Apr 2015 17:58|
2. reading and verifying map
map[ 1] = 0
map[ 2] = 190
map[ 3] = 313
map[ 4] = 360
map[ 5] = 1284
map[ 6] = 1337
map[ 7] = 1755
map[ 8] = 2697
map[ 9] = 4979
map[10] = 5014
map[11] = 5202
map[12] = 5335
map[13] = 5562
map[14] = 5574
verifying map
1 2 3 4 5 6 7 8 9 10 11 12 13 14
3. reading and verifying vartypes
4. reading and verifying varnames
verifying varnames unique
5. reading and verifying sort order
verifying contents
6. reading and verifying display formats
verifying formats
verifying formats correspond to variable type
7. reading and verifying value-label assignment
verifying value-label construction
verifying value label and corresponding variable types
8. reading and verifying variable-label assignment
verifying variable labels construction
9. reading and verifying characteristics
(0 characteristics in file)
10. reading and verifying data
(dtaverify_119 cannot verify that values are correct)
verifying construction
(1 strL variables)
verifying strL construction
11. reading and verifying strLs
verifying construction
(4 strLs expected)
12. reading and verifying value label definitions
verifying construction
(1 labels in file)
no errors or warnings above; .dta file valid
. use stata16_119.dta
. list
+-----------------------------------------------+
| utf8 latin1 ascii utf8_s~l ascii_~l |
|-----------------------------------------------|
1. | ραηδας PÄNDÄS p ραηδας p |
2. | ƤĀńĐąŜ Ö a ƤĀńĐąŜ a |
3. | ᴘᴀᴎᴅᴀS Ü n ᴘᴀᴎᴅᴀS n |
4. | d d |
5. | a a |
|-----------------------------------------------|
6. | s s |
7. | |
+-----------------------------------------------+
. dtaverify stata16_119.dta
(file "stata16_119.dta" is .dta-format 119 from Stata 15, 16, 17, or 18)
1. reading and verifying header
release is 119
byteorder LSF
K (# of vars) is 5
N (# of obs) is 7
label length 0 ||
date length 17 |31 May 2018 23:59|
2. reading and verifying map
map[ 1] = 0
map[ 2] = 160
map[ 3] = 283
map[ 4] = 326
map[ 5] = 992
map[ 6] = 1037
map[ 7] = 1341
map[ 8] = 2025
map[ 9] = 3665
map[10] = 3700
map[11] = 4266
map[12] = 4587
map[13] = 4616
map[14] = 4628
verifying map
1 2 3 4 5 6 7 8 9 10 11 12 13 14
3. reading and verifying vartypes
4. reading and verifying varnames
verifying varnames unique
5. reading and verifying sort order
verifying contents
6. reading and verifying display formats
verifying formats
verifying formats correspond to variable type
7. reading and verifying value-label assignment
verifying value-label construction
verifying value label and corresponding variable types
8. reading and verifying variable-label assignment
verifying variable labels construction
9. reading and verifying characteristics
(0 characteristics in file)
10. reading and verifying data
(dtaverify_119 cannot verify that values are correct)
verifying construction
(2 strL variables)
verifying strL construction
11. reading and verifying strLs
verifying construction
(12 strLs expected)
12. reading and verifying value label definitions
verifying construction
(0 labels in file)
no errors or warnings above; .dta file valid
. use stata16_be_118.dta
. list
+-----------------------------------------------+
| utf8 latin1 ascii utf8_s~l ascii_~l |
|-----------------------------------------------|
1. | ραηδας PÄNDÄS p ραηδας p |
2. | ƤĀńĐąŜ Ö a ƤĀńĐąŜ a |
3. | ᴘᴀᴎᴅᴀS Ü n ᴘᴀᴎᴅᴀS n |
4. | d d |
5. | a a |
|-----------------------------------------------|
6. | s s |
7. | |
+-----------------------------------------------+
. dtaverify stata16_be_118.dta
(file "stata16_be_118.dta" is .dta-format 118 from Stata 14, 15, 16, 17, or 18)
1. reading and verifying header
release is 118
byteorder MSF
K (# of vars) is 5
N (# of obs) is 7
label length 0 ||
date length 17 |31 May 2018 23:59|
2. reading and verifying map
map[ 1] = 0
map[ 2] = 158
map[ 3] = 281
map[ 4] = 324
map[ 5] = 990
map[ 6] = 1023
map[ 7] = 1327
map[ 8] = 2011
map[ 9] = 3651
map[10] = 3686
map[11] = 4252
map[12] = 4573
map[13] = 4602
map[14] = 4614
verifying map
1 2 3 4 5 6 7 8 9 10 11 12 13 14
3. reading and verifying vartypes
4. reading and verifying varnames
verifying varnames unique
5. reading and verifying sort order
verifying contents
6. reading and verifying display formats
verifying formats
verifying formats correspond to variable type
7. reading and verifying value-label assignment
verifying value-label construction
verifying value label and corresponding variable types
8. reading and verifying variable-label assignment
verifying variable labels construction
9. reading and verifying characteristics
(0 characteristics in file)
10. reading and verifying data
(dtaverify_118 cannot verify that values are correct)
verifying construction
(2 strL variables)
verifying strL construction
11. reading and verifying strLs
verifying construction
(12 strLs expected)
12. reading and verifying value label definitions
verifying construction
(0 labels in file)
no errors or warnings above; .dta file valid
. use stata16_be_119.dta
. list
+-----------------------------------------------+
| utf8 latin1 ascii utf8_s~l ascii_~l |
|-----------------------------------------------|
1. | ραηδας PÄNDÄS p ραηδας p |
2. | ƤĀńĐąŜ Ö a ƤĀńĐąŜ a |
3. | ᴘᴀᴎᴅᴀS Ü n ᴘᴀᴎᴅᴀS n |
4. | d d |
5. | a a |
|-----------------------------------------------|
6. | s s |
7. | |
+-----------------------------------------------+
. dtaverify stata16_be_119.dta
(file "stata16_be_119.dta" is .dta-format 119 from Stata 15, 16, 17, or 18)
1. reading and verifying header
release is 119
byteorder MSF
K (# of vars) is 5
N (# of obs) is 7
label length 0 ||
date length 17 |31 May 2018 23:59|
2. reading and verifying map
map[ 1] = 0
map[ 2] = 160
map[ 3] = 283
map[ 4] = 326
map[ 5] = 992
map[ 6] = 1037
map[ 7] = 1341
map[ 8] = 2025
map[ 9] = 3665
map[10] = 3700
map[11] = 4266
map[12] = 4587
map[13] = 4616
map[14] = 4628
verifying map
1 2 3 4 5 6 7 8 9 10 11 12 13 14
3. reading and verifying vartypes
4. reading and verifying varnames
verifying varnames unique
5. reading and verifying sort order
verifying contents
6. reading and verifying display formats
verifying formats
verifying formats correspond to variable type
7. reading and verifying value-label assignment
verifying value-label construction
verifying value label and corresponding variable types
8. reading and verifying variable-label assignment
verifying variable labels construction
9. reading and verifying characteristics
(0 characteristics in file)
10. reading and verifying data
(dtaverify_119 cannot verify that values are correct)
verifying construction
(2 strL variables)
verifying strL construction
11. reading and verifying strLs
verifying construction
(12 strLs expected)
12. reading and verifying value label definitions
verifying construction
(0 labels in file)
no errors or warnings above; .dta file valid