kraken
kraken copied to clipboard
Published
20 hours ago •
mittagessen
mittagessen
[feature request] Training with Metal Performance Shaders
PyTorch recently introduced support for Apple's Metal Performance Shaders (MPS), see https://pytorch.org/blog/introducing-accelerated-pytorch-training-on-mac/.
Using the new mps backend for PyTorch should accelerate training on Apple M1 machines a lot and is a desired feature for Kraken. In this issue I'd like to track the current status.
kraken<4.0.0
Installation in a fresh virtual Python environment:
pip install "kraken<4.0.0"
pip install -U --pre torch torchvision --extra-index-url https://download.pytorch.org/whl/nightly/cpu
Training can be startet with ketos train -f page -t list.eval -d mps, but aborts with a runtime error which does not occur when running with -d cpu:
Traceback (most recent call last):
File "/Users/stweil/src/github/UB-Mannheim/AustrianNewspapers/venv3.9/bin/ketos", line 8, in <module>
sys.exit(cli())
File "/Users/stweil/src/github/UB-Mannheim/AustrianNewspapers/venv3.9/lib/python3.9/site-packages/click/core.py", line 1130, in __call__
return self.main(*args, **kwargs)
File "/Users/stweil/src/github/UB-Mannheim/AustrianNewspapers/venv3.9/lib/python3.9/site-packages/click/core.py", line 1055, in main
rv = self.invoke(ctx)
File "/Users/stweil/src/github/UB-Mannheim/AustrianNewspapers/venv3.9/lib/python3.9/site-packages/click/core.py", line 1657, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/Users/stweil/src/github/UB-Mannheim/AustrianNewspapers/venv3.9/lib/python3.9/site-packages/click/core.py", line 1404, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/Users/stweil/src/github/UB-Mannheim/AustrianNewspapers/venv3.9/lib/python3.9/site-packages/click/core.py", line 760, in invoke
return __callback(*args, **kwargs)
File "/Users/stweil/src/github/UB-Mannheim/AustrianNewspapers/venv3.9/lib/python3.9/site-packages/click/decorators.py", line 26, in new_func
return f(get_current_context(), *args, **kwargs)
File "/Users/stweil/src/github/UB-Mannheim/AustrianNewspapers/venv3.9/lib/python3.9/site-packages/kraken/ketos.py", line 604, in train
trainer.run(_print_eval, _draw_progressbar)
File "/Users/stweil/src/github/UB-Mannheim/AustrianNewspapers/venv3.9/lib/python3.9/site-packages/kraken/lib/train.py", line 481, in run
o = self.model.nn(input, seq_lens)
File "/Users/stweil/src/github/UB-Mannheim/AustrianNewspapers/venv3.9/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl
return forward_call(*input, **kwargs)
File "/Users/stweil/src/github/UB-Mannheim/AustrianNewspapers/venv3.9/lib/python3.9/site-packages/kraken/lib/layers.py", line 27, in forward
inputs = module(*inputs)
File "/Users/stweil/src/github/UB-Mannheim/AustrianNewspapers/venv3.9/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl
return forward_call(*input, **kwargs)
File "/Users/stweil/src/github/UB-Mannheim/AustrianNewspapers/venv3.9/lib/python3.9/site-packages/kraken/lib/layers.py", line 382, in forward
o, _ = self.layer(inputs)
File "/Users/stweil/src/github/UB-Mannheim/AustrianNewspapers/venv3.9/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl
return forward_call(*input, **kwargs)
File "/Users/stweil/src/github/UB-Mannheim/AustrianNewspapers/venv3.9/lib/python3.9/site-packages/torch/nn/modules/rnn.py", line 773, in forward
result = _VF.lstm(input, batch_sizes, hx, self._flat_weights, self.bias,
RuntimeError: start (100) + length (1) exceeds dimension size (100).
kraken>=4.0.0
Latest kraken releases use PyTorch Lightning which currently does not support the mps backend (see https://github.com/PyTorchLightning/pytorch-lightning/issues/13102), so training is not possible with -d mps.
There's now basic MPS support in lightning so acceleration on Mac should work now with the development version.
kraken segmentation currently fails on MacOS with cpu and with mps, so I cannot test the acceleration with mps:
kraken --raise-on-error -d cpu -a -I Bd231_qt_1_10.jpg -o .cpu segment -bl ocr -m austriannewspapers.mlmodel
WARNING:root:Torch version 1.13.0.dev20220829 has not been tested with coremltools. You may run into unexpected errors. Torch 1.11.0 is the most recent version that has been tested.
Loading ANN /Users/stweil/src/github/mittagessen/kraken/kraken/blla.mlmodel ✓
Loading ANN austriannewspapers.mlmodel ✓
Segmenting
╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮
│ /Users/stweil/src/github/mittagessen/kraken/venv3.9/bin/kraken:10 in <module> │
│ │
│ 7 │
│ 8 │
│ 9 if __name__ == "__main__": │
│ ❱ 10 │ sys.exit(cli()) │
│ 11 │
│ │
│ /Users/stweil/src/github/mittagessen/kraken/venv3.9/lib/python3.9/site-packages/click/core.py:11 │
│ 30 in __call__ │
│ │
│ /Users/stweil/src/github/mittagessen/kraken/venv3.9/lib/python3.9/site-packages/click/core.py:10 │
│ 55 in main │
│ │
│ /Users/stweil/src/github/mittagessen/kraken/venv3.9/lib/python3.9/site-packages/click/core.py:16 │
│ 89 in invoke │
│ │
│ /Users/stweil/src/github/mittagessen/kraken/venv3.9/lib/python3.9/site-packages/click/core.py:16 │
│ 26 in _process_result │
│ │
│ /Users/stweil/src/github/mittagessen/kraken/venv3.9/lib/python3.9/site-packages/click/core.py:76 │
│ 0 in invoke │
│ │
│ /Users/stweil/src/github/mittagessen/kraken/kraken/kraken.py:402 in process_pipeline │
│ │
│ 399 │ │ │ for idx, (task, input, output) in enumerate(zip(subcommands, fc, fc[1:])): │
│ 400 │ │ │ │ if len(fc) - 2 == idx: │
│ 401 │ │ │ │ │ ctx.meta['last_process'] = True │
│ ❱ 402 │ │ │ │ task(input=input, output=output) │
│ 403 │ │ #except Exception as e: │
│ 404 │ │ # logger.error(f'Failed processing {io_pair[0]}: {str(e)}') │
│ 405 │ │ # if ctx.meta['raise_failed']: │
│ │
│ /Users/stweil/src/github/mittagessen/kraken/kraken/kraken.py:158 in segmenter │
│ │
│ 155 │ │ │ │ │ │ │ │ pad=pad, │
│ 156 │ │ │ │ │ │ │ │ mask=mask) │
│ 157 │ │ else: │
│ ❱ 158 │ │ │ res = blla.segment(im, text_direction, mask=mask, model=model, device=device │
│ 159 │ except Exception: │
│ 160 │ │ if ctx.meta['raise_failed']: │
│ 161 │ │ │ raise │
│ │
│ /Users/stweil/src/github/mittagessen/kraken/kraken/blla.py:299 in segment │
│ │
│ 296 │ │ │ │ False: 'bottom'}[net.user_metadata['topline']] │
│ 297 │ │ │ logger.debug(f'Baseline location: {loc}') │
│ 298 │ │ rets = compute_segmentation_map(im, mask, net, device) │
│ ❱ 299 │ │ regions = vec_regions(**rets) │
│ 300 │ │ # flatten regions for line ordering/fetch bounding regions │
│ 301 │ │ line_regs = [] │
│ 302 │ │ suppl_obj = [] │
│ │
│ /Users/stweil/src/github/mittagessen/kraken/kraken/blla.py:137 in vec_regions │
│ │
│ 134 │ regions = {} │
│ 135 │ for region_type, idx in cls_map['regions'].items(): │
│ 136 │ │ logger.debug(f'Vectorizing regions of type {region_type}') │
│ ❱ 137 │ │ regions[region_type] = vectorize_regions(heatmap[idx]) │
│ 138 │ for reg_id, regs in regions.items(): │
│ 139 │ │ regions[reg_id] = scale_regions(regs, scale) │
│ 140 │ return regions │
│ │
│ /Users/stweil/src/github/mittagessen/kraken/kraken/lib/segmentation.py:357 in vectorize_regions │
│ │
│ 354 │ if boundaries.type == 'Polygon': │
│ 355 │ │ boundaries = [boundaries.boundary.simplify(10)] │
│ 356 │ else: │
│ ❱ 357 │ │ boundaries = [x.boundary.simplify(10) for x in unary_union(boundaries)] │
│ 358 │ return [np.array(x.coords, dtype=np.uint)[:, [1, 0]].tolist() for x in boundaries] │
│ 359 │
│ 360 │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
TypeError: 'MultiPolygon' object is not iterable
This isn't/shouldn't be Mac OS specific. The latest fix on vec_regions() should have fixed a crash like that but without knowing the shapely version and having the image file I can't really say more.
shapely-2.0a1, Bd231_qt_1_10.jpg. Shapely-1.7.1 fails to install.
With Shapely-1.8.4 and GPU I get errors and kraken does not terminate:
# Running with GPU.
kraken --raise-on-error -d mps -a -I Bd231_qt_1_10.jpg -o .mps segment -bl ocr -m austriannewspapers.mlmodel
WARNING:root:Torch version 1.13.0.dev20220829 has not been tested with coremltools. You may run into unexpected errors. Torch 1.11.0 is the most recent version that has been tested.
Loading ANN /Users/stweil/src/github/mittagessen/kraken/kraken/blla.mlmodel ✓
Loading ANN austriannewspapers.mlmodel ✓
Segmenting [08/29/22 15:32:28] WARNING Boundary tracing failed in baseline elongation: index 1 is out of bounds for axis 0 with size 1 segmentation.py:220
WARNING Boundary tracing failed in baseline elongation: index 1 is out of bounds for axis 0 with size 1 segmentation.py:220
WARNING Boundary tracing failed in baseline elongation: index 1 is out of bounds for axis 0 with size 1 segmentation.py:220
WARNING Boundary tracing failed in baseline elongation: index 1 is out of bounds for axis 0 with size 1 segmentation.py:220
[does not terminate after more than 10 minutes]
Shapely-1.8.4 and CPU works:
# Running with CPU.
kraken --raise-on-error -d cpu -a -I Bd231_qt_1_10.jpg -o .cpu2 segment -bl ocr -m austriannewspapers.mlmodel &
WARNING:root:Torch version 1.13.0.dev20220829 has not been tested with coremltools. You may run into unexpected errors. Torch 1.11.0 is the most recent version that has been tested.
Loading ANN /Users/stweil/src/github/mittagessen/kraken/kraken/blla.mlmodel ✓
Loading ANN austriannewspapers.mlmodel ✓
Segmenting ✓
Processing ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 63/63 0:00:00 0:00:18
Writing recognition results for Bd231_qt_1_10.jpg ✓
Yeah, I'm loath to touch shapely-related issues right now until they stabilise the code base. 1.7.1 should work fine and is pinned now on both the binary packages and environments.
But in this case I was forced to use 1.8.4, see my last comment above. So the pinned 1.7.1 does not always work. I still have to find a solution how to get 1.7.1 installed on MacOS.
In addition the Metal Performance Shaders still don't work as kraken segmentation does not terminate with -d mps.
Latest code with Shapely 1.8.4 no longer hangs, but fails:
(venv3.9) stweil@notebook11 kraken % kraken --raise-on-error -d mps -a -I Bd231_qt_1_10.jpg -o .mps segment -bl ocr -m austriannewspapers.mlmodel
WARNING:root:scikit-learn version 1.1.2 is not supported. Minimum required version: 0.17. Maximum required version: 0.19.2. Disabling scikit-learn conversion API.
WARNING:root:TensorFlow version 2.9.2 has not been tested with coremltools. You may run into unexpected errors. TensorFlow 2.8.0 is the most recent version that has been tested.
WARNING:root:Torch version 1.13.0.dev20220928 has not been tested with coremltools. You may run into unexpected errors. Torch 1.11.0 is the most recent version that has been tested.
Loading ANN /Users/stweil/src/github/mittagessen/kraken/kraken/blla.mlmodel ✓
Loading ANN austriannewspapers.mlmodel ✓
Segmenting ╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮
│ /Users/stweil/src/github/mittagessen/kraken/venv3.9/bin/kraken:10 in <module> │
│ │
│ 7 │
│ 8 │
│ 9 if __name__ == "__main__": │
│ ❱ 10 │ sys.exit(cli()) │
│ 11 │
│ │
│ /Users/stweil/src/github/mittagessen/kraken/venv3.9/lib/python3.9/site-packages/click/core.py:11 │
│ 30 in __call__ │
│ │
│ /Users/stweil/src/github/mittagessen/kraken/venv3.9/lib/python3.9/site-packages/click/core.py:10 │
│ 55 in main │
│ │
│ /Users/stweil/src/github/mittagessen/kraken/venv3.9/lib/python3.9/site-packages/click/core.py:16 │
│ 89 in invoke │
│ │
│ /Users/stweil/src/github/mittagessen/kraken/venv3.9/lib/python3.9/site-packages/click/core.py:16 │
│ 26 in _process_result │
│ │
│ /Users/stweil/src/github/mittagessen/kraken/venv3.9/lib/python3.9/site-packages/click/core.py:76 │
│ 0 in invoke │
│ │
│ /Users/stweil/src/github/mittagessen/kraken/kraken/kraken.py:401 in process_pipeline │
│ │
│ 398 │ │ │ for idx, (task, input, output) in enumerate(zip(subcommands, fc, fc[1:])): │
│ 399 │ │ │ │ if len(fc) - 2 == idx: │
│ 400 │ │ │ │ │ ctx.meta['last_process'] = True │
│ ❱ 401 │ │ │ │ task(input=input, output=output) │
│ 402 │ │ #except Exception as e: │
│ 403 │ │ # logger.error(f'Failed processing {io_pair[0]}: {str(e)}') │
│ 404 │ │ # if ctx.meta['raise_failed']: │
│ │
│ /Users/stweil/src/github/mittagessen/kraken/kraken/kraken.py:159 in segmenter │
│ │
│ 156 │ │ │ │ │ │ │ │ pad=pad, │
│ 157 │ │ │ │ │ │ │ │ mask=mask) │
│ 158 │ │ else: │
│ ❱ 159 │ │ │ res = blla.segment(im, text_direction, mask=mask, model=model, device=device │
│ 160 │ except Exception: │
│ 161 │ │ if ctx.meta['raise_failed']: │
│ 162 │ │ │ raise │
│ │
│ /Users/stweil/src/github/mittagessen/kraken/kraken/blla.py:310 in segment │
│ │
│ 307 │ │ # convert back to net scale │
│ 308 │ │ suppl_obj = scale_regions(suppl_obj, 1/rets['scale']) │
│ 309 │ │ line_regs = scale_regions(line_regs, 1/rets['scale']) │
│ ❱ 310 │ │ lines = vec_lines(**rets, │
│ 311 │ │ │ │ │ │ regions=line_regs, │
│ 312 │ │ │ │ │ │ reading_order_fn=reading_order_fn, │
│ 313 │ │ │ │ │ │ text_direction=text_direction, │
│ │
│ /Users/stweil/src/github/mittagessen/kraken/kraken/blla.py:210 in vec_lines │
│ │
│ 207 │ │ │ if reg_pol.contains(mid_point): │
│ 208 │ │ │ │ suppl_obj.append(regions[reg_idx]) │
│ 209 │ │ │
│ ❱ 210 │ │ pol = calculate_polygonal_environment(baselines=[bl[1]], im_feats=im_feats, supp │
│ 211 │ │ if pol[0] is not None: │
│ 212 │ │ │ lines.append((bl[0], bl[1], pol[0])) │
│ 213 │
│ │
│ /Users/stweil/src/github/mittagessen/kraken/kraken/lib/segmentation.py:701 in │
│ calculate_polygonal_environment │
│ │
│ 698 │ │ │ │
│ 699 │ │ │ env_up, env_bottom = _calc_roi(line, bounds, baselines[:idx] + baselines[idx │
│ 700 │ │ │ │
│ ❱ 701 │ │ │ polygons.append(_extract_patch(env_up, │
│ 702 │ │ │ │ │ │ │ │ │ │ env_bottom, │
│ 703 │ │ │ │ │ │ │ │ │ │ line.astype('int'), │
│ 704 │ │ │ │ │ │ │ │ │ │ offset_line.astype('int'), │
│ │
│ /Users/stweil/src/github/mittagessen/kraken/kraken/lib/segmentation.py:531 in _extract_patch │
│ │
│ 528 │ │ upper_seam = _calc_seam(baseline, upper_polygon, angle, im_feats) │
│ 529 │ │ bottom_seam = _calc_seam(offset_baseline, bottom_offset_polygon, angle, im_feats │
│ 530 │ else: │
│ ❱ 531 │ │ upper_seam = _calc_seam(offset_baseline, upper_offset_polygon, angle, im_feats) │
│ 532 │ │ bottom_seam = _calc_seam(baseline, bottom_polygon, angle, im_feats) │
│ 533 │ │
│ 534 │ upper_seam = geom.LineString(upper_seam).simplify(5) │
│ │
│ /Users/stweil/src/github/mittagessen/kraken/kraken/lib/segmentation.py:462 in _calc_seam │
│ │
│ 459 │ │ │ │ │ │ │ line_seg[0][0], │
│ 460 │ │ │ │ │ │ │ line_seg[1][1], │
│ 461 │ │ │ │ │ │ │ line_seg[1][0]) │
│ ❱ 462 │ │ mask[line_locs] = 0 │
│ 463 │ dist_bias = distance_transform_cdt(mask) │
│ 464 │ # absolute mask │
│ 465 │ mask = np.ones_like(patch, dtype=bool) │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
IndexError: index 0 is out of bounds for axis 0 with size 0
With 2.0a1 it fails, too:
(venv3.9) stweil@notebook11 kraken % kraken --raise-on-error -d mps -a -I Bd231_qt_1_10.jpg -o .mps segment -bl ocr -m austriannewspapers.mlmodel
WARNING:root:scikit-learn version 1.1.2 is not supported. Minimum required version: 0.17. Maximum required version: 0.19.2. Disabling scikit-learn conversion API.
WARNING:root:TensorFlow version 2.9.2 has not been tested with coremltools. You may run into unexpected errors. TensorFlow 2.8.0 is the most recent version that has been tested.
WARNING:root:Torch version 1.13.0.dev20220928 has not been tested with coremltools. You may run into unexpected errors. Torch 1.11.0 is the most recent version that has been tested.
Loading ANN /Users/stweil/src/github/mittagessen/kraken/kraken/blla.mlmodel ✓
Loading ANN austriannewspapers.mlmodel ✓
Segmenting ╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮
│ /Users/stweil/src/github/mittagessen/kraken/venv3.9/bin/kraken:10 in <module> │
│ │
│ 7 │
│ 8 │
│ 9 if __name__ == "__main__": │
│ ❱ 10 │ sys.exit(cli()) │
│ 11 │
│ │
│ /Users/stweil/src/github/mittagessen/kraken/venv3.9/lib/python3.9/site-packages/click/core.py:11 │
│ 30 in __call__ │
│ │
│ /Users/stweil/src/github/mittagessen/kraken/venv3.9/lib/python3.9/site-packages/click/core.py:10 │
│ 55 in main │
│ │
│ /Users/stweil/src/github/mittagessen/kraken/venv3.9/lib/python3.9/site-packages/click/core.py:16 │
│ 89 in invoke │
│ │
│ /Users/stweil/src/github/mittagessen/kraken/venv3.9/lib/python3.9/site-packages/click/core.py:16 │
│ 26 in _process_result │
│ │
│ /Users/stweil/src/github/mittagessen/kraken/venv3.9/lib/python3.9/site-packages/click/core.py:76 │
│ 0 in invoke │
│ │
│ /Users/stweil/src/github/mittagessen/kraken/kraken/kraken.py:401 in process_pipeline │
│ │
│ 398 │ │ │ for idx, (task, input, output) in enumerate(zip(subcommands, fc, fc[1:])): │
│ 399 │ │ │ │ if len(fc) - 2 == idx: │
│ 400 │ │ │ │ │ ctx.meta['last_process'] = True │
│ ❱ 401 │ │ │ │ task(input=input, output=output) │
│ 402 │ │ #except Exception as e: │
│ 403 │ │ # logger.error(f'Failed processing {io_pair[0]}: {str(e)}') │
│ 404 │ │ # if ctx.meta['raise_failed']: │
│ │
│ /Users/stweil/src/github/mittagessen/kraken/kraken/kraken.py:159 in segmenter │
│ │
│ 156 │ │ │ │ │ │ │ │ pad=pad, │
│ 157 │ │ │ │ │ │ │ │ mask=mask) │
│ 158 │ │ else: │
│ ❱ 159 │ │ │ res = blla.segment(im, text_direction, mask=mask, model=model, device=device │
│ 160 │ except Exception: │
│ 161 │ │ if ctx.meta['raise_failed']: │
│ 162 │ │ │ raise │
│ │
│ /Users/stweil/src/github/mittagessen/kraken/kraken/blla.py:299 in segment │
│ │
│ 296 │ │ │ │ False: 'bottom'}[net.user_metadata['topline']] │
│ 297 │ │ │ logger.debug(f'Baseline location: {loc}') │
│ 298 │ │ rets = compute_segmentation_map(im, mask, net, device) │
│ ❱ 299 │ │ regions = vec_regions(**rets) │
│ 300 │ │ # flatten regions for line ordering/fetch bounding regions │
│ 301 │ │ line_regs = [] │
│ 302 │ │ suppl_obj = [] │
│ │
│ /Users/stweil/src/github/mittagessen/kraken/kraken/blla.py:137 in vec_regions │
│ │
│ 134 │ regions = {} │
│ 135 │ for region_type, idx in cls_map['regions'].items(): │
│ 136 │ │ logger.debug(f'Vectorizing regions of type {region_type}') │
│ ❱ 137 │ │ regions[region_type] = vectorize_regions(heatmap[idx]) │
│ 138 │ for reg_id, regs in regions.items(): │
│ 139 │ │ regions[reg_id] = scale_regions(regs, scale) │
│ 140 │ return regions │
│ │
│ /Users/stweil/src/github/mittagessen/kraken/kraken/lib/segmentation.py:360 in vectorize_regions │
│ │
│ 357 │ if boundaries.type == 'Polygon': │
│ 358 │ │ boundaries = [boundaries.boundary.simplify(10)] │
│ 359 │ else: │
│ ❱ 360 │ │ boundaries = [x.boundary.simplify(10) for x in unary_union(boundaries)] │
│ 361 │ return [np.array(x.coords, dtype=np.uint)[:, [1, 0]].tolist() for x in boundaries] │
│ 362 │
│ 363 │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
TypeError: 'MultiPolygon' object is not iterable
Thanks. One is a shapely error (fixed now), the other something else. Unfortunately, I can't reproduce the non-shapely one but it is in the polygonizer and crashes because the RoI has a size of 0 which shouldn't happen.
Shapely 2.0a1 (and same error now with 1.8.4, too):
(venv3.9) stweil@notebook11 kraken % kraken --raise-on-error -d mps -a -I Bd231_qt_1_10.jpg -o .mps segment -bl ocr -m austriannewspapers.mlmodel
WARNING:root:scikit-learn version 1.1.2 is not supported. Minimum required version: 0.17. Maximum required version: 0.19.2. Disabling scikit-learn conversion API.
WARNING:root:TensorFlow version 2.9.2 has not been tested with coremltools. You may run into unexpected errors. TensorFlow 2.8.0 is the most recent version that has been tested.
WARNING:root:Torch version 1.13.0.dev20220928 has not been tested with coremltools. You may run into unexpected errors. Torch 1.11.0 is the most recent version that has been tested.
Loading ANN /Users/stweil/src/github/mittagessen/kraken/kraken/blla.mlmodel ✓
Loading ANN austriannewspapers.mlmodel ✓
Segmenting ╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮
│ /Users/stweil/src/github/mittagessen/kraken/venv3.9/bin/kraken:10 in <module> │
│ │
│ 7 │
│ 8 │
│ 9 if __name__ == "__main__": │
│ ❱ 10 │ sys.exit(cli()) │
│ 11 │
│ │
│ /Users/stweil/src/github/mittagessen/kraken/venv3.9/lib/python3.9/site-packages/click/core.py:11 │
│ 30 in __call__ │
│ │
│ /Users/stweil/src/github/mittagessen/kraken/venv3.9/lib/python3.9/site-packages/click/core.py:10 │
│ 55 in main │
│ │
│ /Users/stweil/src/github/mittagessen/kraken/venv3.9/lib/python3.9/site-packages/click/core.py:16 │
│ 89 in invoke │
│ │
│ /Users/stweil/src/github/mittagessen/kraken/venv3.9/lib/python3.9/site-packages/click/core.py:16 │
│ 26 in _process_result │
│ │
│ /Users/stweil/src/github/mittagessen/kraken/venv3.9/lib/python3.9/site-packages/click/core.py:76 │
│ 0 in invoke │
│ │
│ /Users/stweil/src/github/mittagessen/kraken/kraken/kraken.py:401 in process_pipeline │
│ │
│ 398 │ │ │ for idx, (task, input, output) in enumerate(zip(subcommands, fc, fc[1:])): │
│ 399 │ │ │ │ if len(fc) - 2 == idx: │
│ 400 │ │ │ │ │ ctx.meta['last_process'] = True │
│ ❱ 401 │ │ │ │ task(input=input, output=output) │
│ 402 │ │ #except Exception as e: │
│ 403 │ │ # logger.error(f'Failed processing {io_pair[0]}: {str(e)}') │
│ 404 │ │ # if ctx.meta['raise_failed']: │
│ │
│ /Users/stweil/src/github/mittagessen/kraken/kraken/kraken.py:159 in segmenter │
│ │
│ 156 │ │ │ │ │ │ │ │ pad=pad, │
│ 157 │ │ │ │ │ │ │ │ mask=mask) │
│ 158 │ │ else: │
│ ❱ 159 │ │ │ res = blla.segment(im, text_direction, mask=mask, model=model, device=device │
│ 160 │ except Exception: │
│ 161 │ │ if ctx.meta['raise_failed']: │
│ 162 │ │ │ raise │
│ │
│ /Users/stweil/src/github/mittagessen/kraken/kraken/blla.py:310 in segment │
│ │
│ 307 │ │ # convert back to net scale │
│ 308 │ │ suppl_obj = scale_regions(suppl_obj, 1/rets['scale']) │
│ 309 │ │ line_regs = scale_regions(line_regs, 1/rets['scale']) │
│ ❱ 310 │ │ lines = vec_lines(**rets, │
│ 311 │ │ │ │ │ │ regions=line_regs, │
│ 312 │ │ │ │ │ │ reading_order_fn=reading_order_fn, │
│ 313 │ │ │ │ │ │ text_direction=text_direction, │
│ │
│ /Users/stweil/src/github/mittagessen/kraken/kraken/blla.py:210 in vec_lines │
│ │
│ 207 │ │ │ if reg_pol.contains(mid_point): │
│ 208 │ │ │ │ suppl_obj.append(regions[reg_idx]) │
│ 209 │ │ │
│ ❱ 210 │ │ pol = calculate_polygonal_environment(baselines=[bl[1]], im_feats=im_feats, supp │
│ 211 │ │ if pol[0] is not None: │
│ 212 │ │ │ lines.append((bl[0], bl[1], pol[0])) │
│ 213 │
│ │
│ /Users/stweil/src/github/mittagessen/kraken/kraken/lib/segmentation.py:701 in │
│ calculate_polygonal_environment │
│ │
│ 698 │ │ │ │
│ 699 │ │ │ env_up, env_bottom = _calc_roi(line, bounds, baselines[:idx] + baselines[idx │
│ 700 │ │ │ │
│ ❱ 701 │ │ │ polygons.append(_extract_patch(env_up, │
│ 702 │ │ │ │ │ │ │ │ │ │ env_bottom, │
│ 703 │ │ │ │ │ │ │ │ │ │ line.astype('int'), │
│ 704 │ │ │ │ │ │ │ │ │ │ offset_line.astype('int'), │
│ │
│ /Users/stweil/src/github/mittagessen/kraken/kraken/lib/segmentation.py:531 in _extract_patch │
│ │
│ 528 │ │ upper_seam = _calc_seam(baseline, upper_polygon, angle, im_feats) │
│ 529 │ │ bottom_seam = _calc_seam(offset_baseline, bottom_offset_polygon, angle, im_feats │
│ 530 │ else: │
│ ❱ 531 │ │ upper_seam = _calc_seam(offset_baseline, upper_offset_polygon, angle, im_feats) │
│ 532 │ │ bottom_seam = _calc_seam(baseline, bottom_polygon, angle, im_feats) │
│ 533 │ │
│ 534 │ upper_seam = geom.LineString(upper_seam).simplify(5) │
│ │
│ /Users/stweil/src/github/mittagessen/kraken/kraken/lib/segmentation.py:462 in _calc_seam │
│ │
│ 459 │ │ │ │ │ │ │ line_seg[0][0], │
│ 460 │ │ │ │ │ │ │ line_seg[1][1], │
│ 461 │ │ │ │ │ │ │ line_seg[1][0]) │
│ ❱ 462 │ │ mask[line_locs] = 0 │
│ 463 │ dist_bias = distance_transform_cdt(mask) │
│ 464 │ # absolute mask │
│ 465 │ mask = np.ones_like(patch, dtype=bool) │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
IndexError: index 0 is out of bounds for axis 0 with size 0
I'll have to find an Apple system to figure out why it fails as I can't reproduce it on any of my Linux machines.
The latest code still fails with Shapely-1.8.4 and Shapely-2.0b2, but now with a different error:
(venv3.9) stweil@notebook11 kraken % time kraken --raise-on-error -d mps -a -I Bd231_qt_1_10.jpg -o .mps segment -bl ocr -m austriannewspapers.mlmodel
WARNING:root:scikit-learn version 1.1.2 is not supported. Minimum required version: 0.17. Maximum required version: 0.19.2. Disabling scikit-learn conversion API.
WARNING:root:TensorFlow version 2.9.2 has not been tested with coremltools. You may run into unexpected errors. TensorFlow 2.8.0 is the most recent version that has been tested.
WARNING:root:Torch version 1.13.0.dev20220928 has not been tested with coremltools. You may run into unexpected errors. Torch 1.11.0 is the most recent version that has been tested.
Loading ANN /Users/stweil/src/github/mittagessen/kraken/venv3.9/lib/python3.9/site-packages/kraken/blla.mlmodel ✓
Loading ANN austriannewspapers.mlmodel ✓
Segmenting
╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮
│ /Users/stweil/src/github/mittagessen/kraken/venv3.9/bin/kraken:8 in <module> │
│ │
│ 5 from kraken.kraken import cli │
│ 6 if __name__ == '__main__': │
│ 7 │ sys.argv[0] = re.sub(r'(-script\.pyw|\.exe)?$', '', sys.argv[0]) │
│ ❱ 8 │ sys.exit(cli()) │
│ 9 │
│ │
│ /Users/stweil/src/github/mittagessen/kraken/venv3.9/lib/python3.9/site-packages/click/core.py:11 │
│ 30 in __call__ │
│ │
│ /Users/stweil/src/github/mittagessen/kraken/venv3.9/lib/python3.9/site-packages/click/core.py:10 │
│ 55 in main │
│ │
│ /Users/stweil/src/github/mittagessen/kraken/venv3.9/lib/python3.9/site-packages/click/core.py:16 │
│ 89 in invoke │
│ │
│ /Users/stweil/src/github/mittagessen/kraken/venv3.9/lib/python3.9/site-packages/click/core.py:16 │
│ 26 in _process_result │
│ │
│ /Users/stweil/src/github/mittagessen/kraken/venv3.9/lib/python3.9/site-packages/click/core.py:76 │
│ 0 in invoke │
│ │
│ /Users/stweil/src/github/mittagessen/kraken/venv3.9/lib/python3.9/site-packages/kraken/kraken.py │
│ :401 in process_pipeline │
│ │
│ 398 │ │ │ for idx, (task, input, output) in enumerate(zip(subcommands, fc, fc[1:])): │
│ 399 │ │ │ │ if len(fc) - 2 == idx: │
│ 400 │ │ │ │ │ ctx.meta['last_process'] = True │
│ ❱ 401 │ │ │ │ task(input=input, output=output) │
│ 402 │ │ #except Exception as e: │
│ 403 │ │ # logger.error(f'Failed processing {io_pair[0]}: {str(e)}') │
│ 404 │ │ # if ctx.meta['raise_failed']: │
│ │
│ /Users/stweil/src/github/mittagessen/kraken/venv3.9/lib/python3.9/site-packages/kraken/kraken.py │
│ :159 in segmenter │
│ │
│ 156 │ │ │ │ │ │ │ │ pad=pad, │
│ 157 │ │ │ │ │ │ │ │ mask=mask) │
│ 158 │ │ else: │
│ ❱ 159 │ │ │ res = blla.segment(im, text_direction, mask=mask, model=model, device=device │
│ 160 │ except Exception: │
│ 161 │ │ if ctx.meta['raise_failed']: │
│ 162 │ │ │ raise │
│ │
│ /Users/stweil/src/github/mittagessen/kraken/venv3.9/lib/python3.9/site-packages/kraken/blla.py:3 │
│ 10 in segment │
│ │
│ 307 │ │ # convert back to net scale │
│ 308 │ │ suppl_obj = scale_regions(suppl_obj, 1/rets['scale']) │
│ 309 │ │ line_regs = scale_regions(line_regs, 1/rets['scale']) │
│ ❱ 310 │ │ lines = vec_lines(**rets, │
│ 311 │ │ │ │ │ │ regions=line_regs, │
│ 312 │ │ │ │ │ │ reading_order_fn=reading_order_fn, │
│ 313 │ │ │ │ │ │ text_direction=text_direction, │
│ │
│ /Users/stweil/src/github/mittagessen/kraken/venv3.9/lib/python3.9/site-packages/kraken/blla.py:2 │
│ 10 in vec_lines │
│ │
│ 207 │ │ │ if reg_pol.contains(mid_point): │
│ 208 │ │ │ │ suppl_obj.append(regions[reg_idx]) │
│ 209 │ │ │
│ ❱ 210 │ │ pol = calculate_polygonal_environment(baselines=[bl[1]], im_feats=im_feats, supp │
│ 211 │ │ if pol[0] is not None: │
│ 212 │ │ │ lines.append((bl[0], bl[1], pol[0])) │
│ 213 │
│ │
│ /Users/stweil/src/github/mittagessen/kraken/venv3.9/lib/python3.9/site-packages/kraken/lib/segme │
│ ntation.py:699 in calculate_polygonal_environment │
│ │
│ 696 │ │ │ p_dir = np.mean(np.diff(line.T) * lengths/lengths.sum(), axis=1) │
│ 697 │ │ │ p_dir = (p_dir.T / np.sqrt(np.sum(p_dir**2, axis=-1))) │
│ 698 │ │ │ │
│ ❱ 699 │ │ │ env_up, env_bottom = _calc_roi(line, bounds, baselines[:idx] + baselines[idx │
│ 700 │ │ │ │
│ 701 │ │ │ polygons.append(_extract_patch(env_up, │
│ 702 │ │ │ │ │ │ │ │ │ │ env_bottom, │
│ │
│ /Users/stweil/src/github/mittagessen/kraken/venv3.9/lib/python3.9/site-packages/kraken/lib/segme │
│ ntation.py:608 in _calc_roi │
│ │
│ 605 │ env_bottom = [] │
│ 606 │ # find orthogonal (to linear regression) intersects with adjacent objects to complet │
│ 607 │ for point, upper_bounds_intersect, bottom_bounds_intersect in zip(ip_line, upper_bou │
│ ❱ 608 │ │ upper_limit = _find_closest_point(point, geom.LineString( │
│ 609 │ │ │ [point, upper_bounds_intersect]).intersection(side_a)) │
│ 610 │ │ bottom_limit = _find_closest_point(point, geom.LineString( │
│ 611 │ │ │ [point, bottom_bounds_intersect]).intersection(side_b)) │
│ │
│ /Users/stweil/src/github/mittagessen/kraken/venv3.9/lib/python3.9/site-packages/kraken/lib/segme │
│ ntation.py:602 in _find_closest_point │
│ │
│ 599 │ │ │ else: │
│ 600 │ │ │ │ return nearest_points(spt, t)[1] │
│ 601 │ │ else: │
│ ❱ 602 │ │ │ raise Exception(f'No intersection with boundaries. Shapely intersection obje │
│ 603 │ │
│ 604 │ env_up = [] │
│ 605 │ env_bottom = [] │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
Exception: No intersection with boundaries. Shapely intersection object: LINESTRING (295 3.98529745, 295 1)
kraken --raise-on-error -d mps -a -I Bd231_qt_1_10.jpg -o .mps segment -bl oc 82,13s user 5,33s system 105% cpu 1:23,21 total
Running with CPU and Shapely-2.0b2 now fails, too (regression?):
(venv3.9) stweil@notebook11 kraken % time kraken --raise-on-error -d cpu -a -I Bd231_qt_1_10.jpg -o .cpu segment -bl ocr -m austriannewspapers.mlmodel
WARNING:root:scikit-learn version 1.1.2 is not supported. Minimum required version: 0.17. Maximum required version: 0.19.2. Disabling scikit-learn conversion API.
WARNING:root:TensorFlow version 2.9.2 has not been tested with coremltools. You may run into unexpected errors. TensorFlow 2.8.0 is the most recent version that has been tested.
WARNING:root:Torch version 1.14.0.dev20221104 has not been tested with coremltools. You may run into unexpected errors. Torch 1.11.0 is the most recent version that has been tested.
Loading ANN /Users/stweil/src/github/mittagessen/kraken/venv3.9/lib/python3.9/site-packages/kraken/blla.mlmodel ✓
Loading ANN austriannewspapers.mlmodel ✓
Segmenting
╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮
│ /Users/stweil/src/github/mittagessen/kraken/venv3.9/bin/kraken:8 in <module> │
│ │
│ 5 from kraken.kraken import cli │
│ 6 if __name__ == '__main__': │
│ 7 │ sys.argv[0] = re.sub(r'(-script\.pyw|\.exe)?$', '', sys.argv[0]) │
│ ❱ 8 │ sys.exit(cli()) │
│ 9 │
│ │
│ /Users/stweil/src/github/mittagessen/kraken/venv3.9/lib/python3.9/site-packages/click/core.py:11 │
│ 30 in __call__ │
│ │
│ /Users/stweil/src/github/mittagessen/kraken/venv3.9/lib/python3.9/site-packages/click/core.py:10 │
│ 55 in main │
│ │
│ /Users/stweil/src/github/mittagessen/kraken/venv3.9/lib/python3.9/site-packages/click/core.py:16 │
│ 89 in invoke │
│ │
│ /Users/stweil/src/github/mittagessen/kraken/venv3.9/lib/python3.9/site-packages/click/core.py:16 │
│ 26 in _process_result │
│ │
│ /Users/stweil/src/github/mittagessen/kraken/venv3.9/lib/python3.9/site-packages/click/core.py:76 │
│ 0 in invoke │
│ │
│ /Users/stweil/src/github/mittagessen/kraken/venv3.9/lib/python3.9/site-packages/kraken/kraken.py │
│ :401 in process_pipeline │
│ │
│ 398 │ │ │ for idx, (task, input, output) in enumerate(zip(subcommands, fc, fc[1:])): │
│ 399 │ │ │ │ if len(fc) - 2 == idx: │
│ 400 │ │ │ │ │ ctx.meta['last_process'] = True │
│ ❱ 401 │ │ │ │ task(input=input, output=output) │
│ 402 │ │ #except Exception as e: │
│ 403 │ │ # logger.error(f'Failed processing {io_pair[0]}: {str(e)}') │
│ 404 │ │ # if ctx.meta['raise_failed']: │
│ │
│ /Users/stweil/src/github/mittagessen/kraken/venv3.9/lib/python3.9/site-packages/kraken/kraken.py │
│ :159 in segmenter │
│ │
│ 156 │ │ │ │ │ │ │ │ pad=pad, │
│ 157 │ │ │ │ │ │ │ │ mask=mask) │
│ 158 │ │ else: │
│ ❱ 159 │ │ │ res = blla.segment(im, text_direction, mask=mask, model=model, device=device │
│ 160 │ except Exception: │
│ 161 │ │ if ctx.meta['raise_failed']: │
│ 162 │ │ │ raise │
│ │
│ /Users/stweil/src/github/mittagessen/kraken/venv3.9/lib/python3.9/site-packages/kraken/blla.py:3 │
│ 10 in segment │
│ │
│ 307 │ │ # convert back to net scale │
│ 308 │ │ suppl_obj = scale_regions(suppl_obj, 1/rets['scale']) │
│ 309 │ │ line_regs = scale_regions(line_regs, 1/rets['scale']) │
│ ❱ 310 │ │ lines = vec_lines(**rets, │
│ 311 │ │ │ │ │ │ regions=line_regs, │
│ 312 │ │ │ │ │ │ reading_order_fn=reading_order_fn, │
│ 313 │ │ │ │ │ │ text_direction=text_direction, │
│ │
│ /Users/stweil/src/github/mittagessen/kraken/venv3.9/lib/python3.9/site-packages/kraken/blla.py:2 │
│ 10 in vec_lines │
│ │
│ 207 │ │ │ if reg_pol.contains(mid_point): │
│ 208 │ │ │ │ suppl_obj.append(regions[reg_idx]) │
│ 209 │ │ │
│ ❱ 210 │ │ pol = calculate_polygonal_environment(baselines=[bl[1]], im_feats=im_feats, supp │
│ 211 │ │ if pol[0] is not None: │
│ 212 │ │ │ lines.append((bl[0], bl[1], pol[0])) │
│ 213 │
│ │
│ /Users/stweil/src/github/mittagessen/kraken/venv3.9/lib/python3.9/site-packages/kraken/lib/segme │
│ ntation.py:701 in calculate_polygonal_environment │
│ │
│ 698 │ │ │ │
│ 699 │ │ │ env_up, env_bottom = _calc_roi(line, bounds, baselines[:idx] + baselines[idx │
│ 700 │ │ │ │
│ ❱ 701 │ │ │ polygons.append(_extract_patch(env_up, │
│ 702 │ │ │ │ │ │ │ │ │ │ env_bottom, │
│ 703 │ │ │ │ │ │ │ │ │ │ line.astype('int'), │
│ 704 │ │ │ │ │ │ │ │ │ │ offset_line.astype('int'), │
│ │
│ /Users/stweil/src/github/mittagessen/kraken/venv3.9/lib/python3.9/site-packages/kraken/lib/segme │
│ ntation.py:551 in _extract_patch │
│ │
│ 548 │ # offsetting might produce bounds outside the image. Clip it to the image bounds. │
│ 549 │ polygon = np.concatenate(([end_points[0]], upper_seam, [end_points[-1]], bottom_seam │
│ 550 │ polygon = geom.Polygon(polygon) │
│ ❱ 551 │ polygon = np.array(roi_polygon.intersection(polygon).boundary.coords, dtype=int) │
│ 552 │ return polygon │
│ 553 │
│ 554 │
│ │
│ /Users/stweil/src/github/mittagessen/kraken/venv3.9/lib/python3.9/site-packages/shapely/geometry │
│ /base.py:570 in intersection │
│ │
│ 567 │ │ │
│ 568 │ │ Refer to `shapely.intersection` for full documentation. │
│ 569 │ │ """ │
│ ❱ 570 │ │ return shapely.intersection(self, other, grid_size=grid_size) │
│ 571 │ │
│ 572 │ def symmetric_difference(self, other, grid_size=None): │
│ 573 │ │ """ │
│ │
│ /Users/stweil/src/github/mittagessen/kraken/venv3.9/lib/python3.9/site-packages/shapely/decorato │
│ rs.py:77 in wrapped │
│ │
│ 74 │ │ try: │
│ 75 │ │ │ for arr in array_args: │
│ 76 │ │ │ │ arr.flags.writeable = False │
│ ❱ 77 │ │ │ return func(*args, **kwargs) │
│ 78 │ │ finally: │
│ 79 │ │ │ for arr, old_flag in zip(array_args, old_flags): │
│ 80 │ │ │ │ arr.flags.writeable = old_flag │
│ │
│ /Users/stweil/src/github/mittagessen/kraken/venv3.9/lib/python3.9/site-packages/shapely/set_oper │
│ ations.py:133 in intersection │
│ │
│ 130 │ │ │
│ 131 │ │ return lib.intersection_prec(a, b, grid_size, **kwargs) │
│ 132 │ │
│ ❱ 133 │ return lib.intersection(a, b, **kwargs) │
│ 134 │
│ 135 │
│ 136 @multithreading_enabled │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
GEOSException: TopologyException: Input geom 1 is invalid: Self-intersection at 1343 290
kraken --raise-on-error -d cpu -a -I Bd231_qt_1_10.jpg -o .mps segment -bl oc 18,49s user 2,90s system 132% cpu 16,140 total
The same test works with Shapely 1.8.5.post1.
My latest attempt to train using MPS failed because PyTorch doesn't yet support CTC layers in MPS. The error message referred to a PyTorch issue, inviting to ask for support of specific features.
Update: latest pytorch still has the same problem:
NotImplementedError: The operator 'aten::_ctc_loss' is not currently implemented for the MPS device. If you want this op to be added in priority during the prototype
phase of this feature, please comment on https://github.com/pytorch/pytorch/issues/77764. As a temporary fix, you can set the environment variable
`PYTORCH_ENABLE_MPS_FALLBACK=1` to use the CPU as a fallback for this op. WARNING: this will be slower than running natively on MPS.
With PYTORCH_ENABLE_MPS_FALLBACK=1 training seems to work, but it is much slower than a training with CPU only:
% ketos train -f page -t list.eval -d mps
GPU available: True (mps), used: True
TPU available: False, using: 0 TPU cores
IPU available: False, using: 0 IPUs
HPU available: False, using: 0 HPUs
`Trainer(val_check_interval=1.0)` was configured so validation will run at the end of the training epoch..
┏━━━━┳━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ ┃ Name ┃ Type ┃ Params ┃ In sizes ┃ Out sizes ┃
┡━━━━╇━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ 0 │ val_cer │ CharErrorRate │ 0 │ ? │ ? │
│ 1 │ val_wer │ WordErrorRate │ 0 │ ? │ ? │
│ 2 │ net │ MultiParamSequential │ 4.0 M │ [[1, 1, 120, 400], '?'] │ [[1, 121, 1, 50], '?'] │
│ 3 │ net.C_0 │ ActConv2D │ 1.3 K │ [[1, 1, 120, 400], '?'] │ [[1, 32, 120, 400], '?'] │
│ 4 │ net.Do_1 │ Dropout │ 0 │ [[1, 32, 120, 400], '?'] │ [[1, 32, 120, 400], '?'] │
│ 5 │ net.Mp_2 │ MaxPool │ 0 │ [[1, 32, 120, 400], '?'] │ [[1, 32, 60, 200], '?'] │
│ 6 │ net.C_3 │ ActConv2D │ 40.0 K │ [[1, 32, 60, 200], '?'] │ [[1, 32, 60, 200], '?'] │
│ 7 │ net.Do_4 │ Dropout │ 0 │ [[1, 32, 60, 200], '?'] │ [[1, 32, 60, 200], '?'] │
│ 8 │ net.Mp_5 │ MaxPool │ 0 │ [[1, 32, 60, 200], '?'] │ [[1, 32, 30, 100], '?'] │
│ 9 │ net.C_6 │ ActConv2D │ 55.4 K │ [[1, 32, 30, 100], '?'] │ [[1, 64, 30, 100], '?'] │
│ 10 │ net.Do_7 │ Dropout │ 0 │ [[1, 64, 30, 100], '?'] │ [[1, 64, 30, 100], '?'] │
│ 11 │ net.Mp_8 │ MaxPool │ 0 │ [[1, 64, 30, 100], '?'] │ [[1, 64, 15, 50], '?'] │
│ 12 │ net.C_9 │ ActConv2D │ 110 K │ [[1, 64, 15, 50], '?'] │ [[1, 64, 15, 50], '?'] │
│ 13 │ net.Do_10 │ Dropout │ 0 │ [[1, 64, 15, 50], '?'] │ [[1, 64, 15, 50], '?'] │
│ 14 │ net.S_11 │ Reshape │ 0 │ [[1, 64, 15, 50], '?'] │ [[1, 960, 1, 50], '?'] │
│ 15 │ net.L_12 │ TransposedSummarizingRNN │ 1.9 M │ [[1, 960, 1, 50], '?'] │ [[1, 400, 1, 50], '?'] │
│ 16 │ net.Do_13 │ Dropout │ 0 │ [[1, 400, 1, 50], '?'] │ [[1, 400, 1, 50], '?'] │
│ 17 │ net.L_14 │ TransposedSummarizingRNN │ 963 K │ [[1, 400, 1, 50], '?'] │ [[1, 400, 1, 50], '?'] │
│ 18 │ net.Do_15 │ Dropout │ 0 │ [[1, 400, 1, 50], '?'] │ [[1, 400, 1, 50], '?'] │
│ 19 │ net.L_16 │ TransposedSummarizingRNN │ 963 K │ [[1, 400, 1, 50], '?'] │ [[1, 400, 1, 50], '?'] │
│ 20 │ net.Do_17 │ Dropout │ 0 │ [[1, 400, 1, 50], '?'] │ [[1, 400, 1, 50], '?'] │
│ 21 │ net.O_18 │ LinSoftmax │ 48.5 K │ [[1, 400, 1, 50], '?'] │ [[1, 121, 1, 50], '?'] │
└────┴───────────┴──────────────────────────┴────────┴──────────────────────────┴──────────────────────────┘
Trainable params: 4.0 M
Non-trainable params: 0
Total params: 4.0 M
Total estimated model params size (MB): 16
stage 0/∞ ━╸━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 175/4514 0:09:17 • 2:45:22 0.44it/s early_stopping: 0/10 -inf
Update: ketos train -f page -t list.eval -d mps still complains about the missing MPS implementation of aten::_ctc_loss.
@mittagessen, I suggest to re-open this feature request because MPS support is still incomplete (not sufficient for a training).
Related pytorch issue: https://github.com/pytorch/pytorch/issues/77764#issuecomment-1304440087.