WoWDBDefs icon indicating copy to clipboard operation
WoWDBDefs copied to clipboard

DBD to SQL converter script

Open Smerdokryl opened this issue 2 years ago • 4 comments

Script for creating SQL table structure statements from .dbd files. My intention would be to use it as a precursor to CSV import, because on its own the latter doesn't create any indices or foreign keys.

TODO items:

  • Do something about self-referencing columns such as SpellVisualMissile.SpellVisualMissileSetID which sometimes cause FK errors on creation even with FOREIGN_KEY_CHECKS=0, so I left them as simple indices for now.
  • Include field comments in the tables.
  • Maybe make the script connect to DB and execute directly instead of dumping files.

Smerdokryl avatar Feb 11 '23 08:02 Smerdokryl

Maybe make the script connect to DB and execute directly instead of dumping files.

I'd suggest to instead have --output default to sys.stdout to allow for piping the output, keeping tools single purpose and standalone: ./dbd_to_sql.py --build 3.3.5.12340 | mysql -D mydatabase.

bloerwald avatar Feb 11 '23 10:02 bloerwald

Maybe make the script connect to DB and execute directly instead of dumping files.

I'd suggest to instead have --output default to sys.stdout to allow for piping the output, keeping tools single purpose and standalone: ./dbd_to_sql.py --build 3.3.5.12340 | mysql -D mydatabase.

Great idea! What about the diagnostic prints - redirect them to stderr or just disable with this default --output?

Smerdokryl avatar Feb 11 '23 23:02 Smerdokryl

Unix tools usually just have no output unless there is an error, which is printed to stderr, so the question vanishes. If you do want to have unconditional debug output, stderr still seems to be the right choice.

The implementation looks fine. Another standard which I forgot to mention, which also avoids the "is the variable a file or a string?" thing is to have "-" mean "use stdout" and default to that, i.e.

-from io import TextIOWrapper
 from sys import stdout, stderr

 parser = ArgumentParser();
 group = parser.add_mutually_exclusive_group();
 group.add_argument('--layout', type=str, help="target layout, e.g. '90747013'");
 group.add_argument('--build', type=str, help="target build, e.g. '10.0.0.43342'");
 parser.add_argument('dbds', type=str, nargs='*', help='directory with / list of for dbd files to process');
-parser.add_argument('--output', type=str, default=stdout, help='file to dump sql to');
+parser.add_argument('--output', type=str, default='-', help='file to dump sql to');
 args = parser.parse_args();

 dbds:list[str] = args.dbds or os.path.join(
 	os.path.dirname( # WoWDBDefs/
 	os.path.dirname( # code/
 	os.path.dirname( # Python/
 		os.path.abspath(__file__) # ./dbd_to_sql.py
 	))),
 	'definitions'
 );
 if not dbds[0].endswith(dbd.file_suffix):
 	dbds = glob(os.path.join(dbds[0], '*.dbd'));

-outfile:TextIOWrapper = args.output;
-if type(outfile) != TextIOWrapper:
-	outfile = open(outfile, 'a');
+outfile = sys.stdout if args.output == '-' else open(outfile, 'a');

bloerwald avatar Feb 12 '23 14:02 bloerwald

The script uses a global variable keys to store primary key information. Consider passing this information as a parameter to functions instead of using a global variable.

manuelaidos123 avatar Dec 09 '23 11:12 manuelaidos123