nerfies
nerfies copied to clipboard
Published
20 hours ago •
google
google
Cannot run eval.py
Hi, Nerfies team.
Thank you for creating such great work!
I cannot run eval.py. However, train.py is working properly.
Any suggestion to solve the problem? Thank you.
Configuration:
include 'gpu_fullhd.gin'
TrainConfig.use_background_loss = False
ModelConfig.num_coarse_samples = 64
ModelConfig.num_fine_samples = 128
Terminal output:
I0520 14:04:02.444777 140460358084416 eval.py:202] *** Starting experiment
I0520 14:04:02.445011 140460358084416 eval.py:205] *** Loading Gin configs from: ['configs/train.gin']
I0520 14:04:02.448294 140460358084416 resource_reader.py:50] system_path_file_exists:gpu_fullhd.gin
E0520 14:04:02.448478 140460358084416 resource_reader.py:55] Path not found: gpu_fullhd.gin
I0520 14:04:02.449614 140460358084416 resource_reader.py:50] system_path_file_exists:warp_defaults.gin
E0520 14:04:02.449765 140460358084416 resource_reader.py:55] Path not found: warp_defaults.gin
I0520 14:04:02.462348 140460358084416 resource_reader.py:50] system_path_file_exists:defaults.gin
E0520 14:04:02.462505 140460358084416 resource_reader.py:55] Path not found: defaults.gin
I0520 14:04:02.472962 140460358084416 eval.py:221] exp_dir = runs/labjuice
I0520 14:04:02.474851 140460358084416 eval.py:226] summary_dir = runs/labjuice/summaries/eval
I0520 14:04:02.475837 140460358084416 eval.py:231] renders_dir = runs/labjuice/renders
I0520 14:04:02.476572 140460358084416 eval.py:236] checkpoint_dir = runs/labjuice/checkpoints
I0520 14:04:04.813610 140460358084416 eval.py:250] Creating datasource: {'type': 'nerfies', 'data_dir': 'data/labjuice'}
I0520 14:04:04.858571 140460358084416 nerfies.py:73] *** Loading dataset IDs from data/labjuice/dataset.json
I0520 14:04:04.924658 140460358084416 core.py:302] *** Creating a dataset with 11 items.
I0520 14:04:07.831447 140417687979776 core.py:375] Loaded item frame_0120: shape=(567, 1008, 3), scale_factor=1.000000, metadata={'appearance': 119, 'warp': 119}
I0520 14:04:07.845319 140417704765184 core.py:375] Loaded item frame_0061: shape=(567, 1008, 3), scale_factor=1.000000, metadata={'appearance': 60, 'warp': 60}
I0520 14:04:07.849324 140417035134720 core.py:375] Loaded item frame_0269: shape=(567, 1008, 3), scale_factor=1.000000, metadata={'appearance': 268, 'warp': 268}
I0520 14:04:07.849992 140417679587072 core.py:375] Loaded item frame_0150: shape=(567, 1008, 3), scale_factor=1.000000, metadata={'appearance': 149, 'warp': 149}
I0520 14:04:07.852888 140417671194368 core.py:375] Loaded item frame_0180: shape=(567, 1008, 3), scale_factor=1.000000, metadata={'appearance': 179, 'warp': 179}
I0520 14:04:07.853951 140449278314240 core.py:375] Loaded item frame_0002: shape=(567, 1008, 3), scale_factor=1.000000, metadata={'appearance': 1, 'warp': 1}
I0520 14:04:07.855388 140417662801664 core.py:375] Loaded item frame_0210: shape=(567, 1008, 3), scale_factor=1.000000, metadata={'appearance': 209, 'warp': 209}
I0520 14:04:07.855896 140449269921536 core.py:375] Loaded item frame_0031: shape=(567, 1008, 3), scale_factor=1.000000, metadata={'appearance': 30, 'warp': 30}
I0520 14:04:07.856396 140417043527424 core.py:375] Loaded item frame_0239: shape=(567, 1008, 3), scale_factor=1.000000, metadata={'appearance': 238, 'warp': 238}
I0520 14:04:07.857960 140417026742016 core.py:375] Loaded item frame_0299: shape=(567, 1008, 3), scale_factor=1.000000, metadata={'appearance': 298, 'warp': 298}
I0520 14:04:07.859418 140417696372480 core.py:375] Loaded item frame_0091: shape=(567, 1008, 3), scale_factor=1.000000, metadata={'appearance': 90, 'warp': 90}
I0520 14:04:08.227674 140460358084416 core.py:138] *** Loaded dataset items: num_rays=6286896, num_examples=11
I0520 14:04:08.655340 140460358084416 core.py:302] *** Creating a dataset with 38 items.
I0520 14:04:09.231013 140416028698368 core.py:375] Loaded item frame_0001: shape=(567, 1008, 3), scale_factor=1.000000, metadata={'appearance': 0, 'warp': 0}
I0520 14:04:09.267059 140413033805568 core.py:375] Loaded item frame_0049: shape=(567, 1008, 3), scale_factor=1.000000, metadata={'appearance': 48, 'warp': 48}
I0520 14:04:09.290294 140411976791808 core.py:375] Loaded item frame_0105: shape=(567, 1008, 3), scale_factor=1.000000, metadata={'appearance': 104, 'warp': 104}
I0520 14:04:09.391631 140411993577216 core.py:375] Loaded item frame_0089: shape=(567, 1008, 3), scale_factor=1.000000, metadata={'appearance': 88, 'warp': 88}
I0520 14:04:09.405202 140410332641024 core.py:375] Loaded item frame_0241: shape=(567, 1008, 3), scale_factor=1.000000, metadata={'appearance': 240, 'warp': 240}
I0520 14:04:09.407492 140416020305664 core.py:375] Loaded item frame_0009: shape=(567, 1008, 3), scale_factor=1.000000, metadata={'appearance': 8, 'warp': 8}
I0520 14:04:09.410925 140415995127552 core.py:375] Loaded item frame_0033: shape=(567, 1008, 3), scale_factor=1.000000, metadata={'appearance': 32, 'warp': 32}
I0520 14:04:09.417380 140416011912960 core.py:375] Loaded item frame_0017: shape=(567, 1008, 3), scale_factor=1.000000, metadata={'appearance': 16, 'warp': 16}
I0520 14:04:09.426120 140412001969920 core.py:375] Loaded item frame_0081: shape=(567, 1008, 3), scale_factor=1.000000, metadata={'appearance': 80, 'warp': 80}
I0520 14:04:09.440179 140409795770112 core.py:375] Loaded item frame_0297: shape=(567, 1008, 3), scale_factor=1.000000, metadata={'appearance': 296, 'warp': 296}
I0520 14:04:09.446149 140410827548416 core.py:375] Loaded item frame_0225: shape=(567, 1008, 3), scale_factor=1.000000, metadata={'appearance': 224, 'warp': 224}
I0520 14:04:09.452231 140411985184512 core.py:375] Loaded item frame_0097: shape=(567, 1008, 3), scale_factor=1.000000, metadata={'appearance': 96, 'warp': 96}
I0520 14:04:09.478314 140416003520256 core.py:375] Loaded item frame_0025: shape=(567, 1008, 3), scale_factor=1.000000, metadata={'appearance': 24, 'warp': 24}
I0520 14:04:09.483593 140411372812032 core.py:375] Loaded item frame_0161: shape=(567, 1008, 3), scale_factor=1.000000, metadata={'appearance': 160, 'warp': 160}
I0520 14:04:09.488675 140412010362624 core.py:375] Loaded item frame_0073: shape=(567, 1008, 3), scale_factor=1.000000, metadata={'appearance': 72, 'warp': 72}
I0520 14:04:09.489866 140411356026624 core.py:375] Loaded item frame_0177: shape=(567, 1008, 3), scale_factor=1.000000, metadata={'appearance': 176, 'warp': 176}
I0520 14:04:09.501473 140411406382848 core.py:375] Loaded item frame_0129: shape=(567, 1008, 3), scale_factor=1.000000, metadata={'appearance': 128, 'warp': 128}
I0520 14:04:09.502885 140410869511936 core.py:375] Loaded item frame_0185: shape=(567, 1008, 3), scale_factor=1.000000, metadata={'appearance': 184, 'warp': 184}
I0520 14:04:09.504619 140413025412864 core.py:375] Loaded item frame_0057: shape=(567, 1008, 3), scale_factor=1.000000, metadata={'appearance': 56, 'warp': 56}
I0520 14:04:09.504841 140411960006400 core.py:375] Loaded item frame_0121: shape=(567, 1008, 3), scale_factor=1.000000, metadata={'appearance': 120, 'warp': 120}
I0520 14:04:09.505058 140410282284800 core.py:375] Loaded item frame_0289: shape=(567, 1008, 3), scale_factor=1.000000, metadata={'appearance': 288, 'warp': 288}
I0520 14:04:09.506297 140410290677504 core.py:375] Loaded item frame_0281: shape=(567, 1008, 3), scale_factor=1.000000, metadata={'appearance': 280, 'warp': 280}
I0520 14:04:09.506653 140411381204736 core.py:375] Loaded item frame_0153: shape=(567, 1008, 3), scale_factor=1.000000, metadata={'appearance': 152, 'warp': 152}
I0520 14:04:09.509786 140410835941120 core.py:375] Loaded item frame_0217: shape=(567, 1008, 3), scale_factor=1.000000, metadata={'appearance': 216, 'warp': 216}
I0520 14:04:09.516040 140411364419328 core.py:375] Loaded item frame_0169: shape=(567, 1008, 3), scale_factor=1.000000, metadata={'appearance': 168, 'warp': 168}
I0520 14:04:09.518889 140410844333824 core.py:375] Loaded item frame_0209: shape=(567, 1008, 3), scale_factor=1.000000, metadata={'appearance': 208, 'warp': 208}
I0520 14:04:09.520003 140410299070208 core.py:375] Loaded item frame_0273: shape=(567, 1008, 3), scale_factor=1.000000, metadata={'appearance': 272, 'warp': 272}
I0520 14:04:09.522838 140411389597440 core.py:375] Loaded item frame_0145: shape=(567, 1008, 3), scale_factor=1.000000, metadata={'appearance': 144, 'warp': 144}
I0520 14:04:09.523147 140411968399104 core.py:375] Loaded item frame_0113: shape=(567, 1008, 3), scale_factor=1.000000, metadata={'appearance': 112, 'warp': 112}
I0520 14:04:09.523772 140410861119232 core.py:375] Loaded item frame_0193: shape=(567, 1008, 3), scale_factor=1.000000, metadata={'appearance': 192, 'warp': 192}
I0520 14:04:09.525799 140413436401408 core.py:375] Loaded item frame_0041: shape=(567, 1008, 3), scale_factor=1.000000, metadata={'appearance': 40, 'warp': 40}
I0520 14:04:09.526642 140410307462912 core.py:375] Loaded item frame_0265: shape=(567, 1008, 3), scale_factor=1.000000, metadata={'appearance': 264, 'warp': 264}
I0520 14:04:09.528127 140410324248320 core.py:375] Loaded item frame_0249: shape=(567, 1008, 3), scale_factor=1.000000, metadata={'appearance': 248, 'warp': 248}
I0520 14:04:09.528872 140410852726528 core.py:375] Loaded item frame_0201: shape=(567, 1008, 3), scale_factor=1.000000, metadata={'appearance': 200, 'warp': 200}
I0520 14:04:09.531115 140412622956288 core.py:375] Loaded item frame_0065: shape=(567, 1008, 3), scale_factor=1.000000, metadata={'appearance': 64, 'warp': 64}
I0520 14:04:09.533269 140410819155712 core.py:375] Loaded item frame_0233: shape=(567, 1008, 3), scale_factor=1.000000, metadata={'appearance': 232, 'warp': 232}
I0520 14:04:09.535274 140411397990144 core.py:375] Loaded item frame_0137: shape=(567, 1008, 3), scale_factor=1.000000, metadata={'appearance': 136, 'warp': 136}
I0520 14:04:09.535899 140410315855616 core.py:375] Loaded item frame_0257: shape=(567, 1008, 3), scale_factor=1.000000, metadata={'appearance': 256, 'warp': 256}
I0520 14:04:10.983216 140460358084416 core.py:138] *** Loaded dataset items: num_rays=21718368, num_examples=38
W0520 14:04:12.018282 140460358084416 nerfies.py:170] test camera path does not exist: data/labjuice/camera-paths/orbit-extreme
I0520 14:04:45.636032 140460358084416 checkpoints.py:152] Restoring checkpoint from runs/labjuice/checkpoints/checkpoint_340000
I0520 14:04:46.048658 140460358084416 eval.py:157] [train:1/11] Processing frame_0002
I0520 14:04:46.049307 140460358084416 evaluation.py:55] Rendering ray batch: 0/571536
Traceback (most recent call last):
File "eval.py", line 378, in <module>
app.run(main)
File "/ist/users/pakkaponp/.conda/envs/jax/lib/python3.6/site-packages/absl/app.py", line 303, in run
_run_main(main, args)
File "/ist/users/pakkaponp/.conda/envs/jax/lib/python3.6/site-packages/absl/app.py", line 251, in _run_main
sys.exit(main(argv))
File "eval.py", line 347, in main
datasource=datasource)
File "eval.py", line 188, in process_iterator
datasource=datasource)
File "eval.py", line 73, in process_batch
rgb, depth_exp, depth_med, acc = render_fn(state, batch, rng=rng)
File "/ist/ist-share/vision/pakkapon/nerf_proj/nerfies/nerfies/evaluation.py", line 80, in render_image
state.warp_alpha)
File "/ist/users/pakkaponp/.conda/envs/jax/lib/python3.6/site-packages/jax/api.py", line 1571, in f_pmapped
global_arg_shapes=tuple(global_arg_shapes_flat))
File "/ist/users/pakkaponp/.conda/envs/jax/lib/python3.6/site-packages/jax/core.py", line 1461, in bind
return call_bind(self, fun, *args, **params)
File "/ist/users/pakkaponp/.conda/envs/jax/lib/python3.6/site-packages/jax/core.py", line 1393, in call_bind
outs = primitive.process(top_trace, fun, tracers, params)
File "/ist/users/pakkaponp/.conda/envs/jax/lib/python3.6/site-packages/jax/core.py", line 1464, in process
return trace.process_map(self, fun, tracers, params)
File "/ist/users/pakkaponp/.conda/envs/jax/lib/python3.6/site-packages/jax/core.py", line 600, in process_call
return primitive.impl(f, *tracers, **params)
File "/ist/users/pakkaponp/.conda/envs/jax/lib/python3.6/site-packages/jax/interpreters/pxla.py", line 618, in xla_pmap_impl
*abstract_args)
File "/ist/users/pakkaponp/.conda/envs/jax/lib/python3.6/site-packages/jax/linear_util.py", line 260, in memoized_fun
ans = call(fun, *args)
File "/ist/users/pakkaponp/.conda/envs/jax/lib/python3.6/site-packages/jax/interpreters/pxla.py", line 688, in parallel_callable
jaxpr, out_sharded_avals, consts = pe.trace_to_jaxpr_final(fun, global_sharded_avals, transform_name="pmap")
File "/ist/users/pakkaponp/.conda/envs/jax/lib/python3.6/site-packages/jax/interpreters/partial_eval.py", line 1209, in trace_to_jaxpr_final
jaxpr, out_avals, consts = trace_to_subjaxpr_dynamic(fun, main, in_avals)
File "/ist/users/pakkaponp/.conda/envs/jax/lib/python3.6/site-packages/jax/interpreters/partial_eval.py", line 1188, in trace_to_subjaxpr_dynamic
ans = fun.call_wrapped(*in_tracers)
File "/ist/users/pakkaponp/.conda/envs/jax/lib/python3.6/site-packages/jax/linear_util.py", line 166, in call_wrapped
ans = self.f(*args, **dict(self.params, **kwargs))
File "eval.py", line 303, in _model_fn
mutable=False)
File "/ist/users/pakkaponp/.conda/envs/jax/lib/python3.6/site-packages/flax/linen/module.py", line 686, in apply
return apply(fn, mutable=mutable)(variables, rngs=rngs)
File "/ist/users/pakkaponp/.conda/envs/jax/lib/python3.6/site-packages/flax/core/scope.py", line 591, in wrapper
y = fn(root, *args, **kwargs)
File "/ist/users/pakkaponp/.conda/envs/jax/lib/python3.6/site-packages/flax/linen/module.py", line 685, in <lambda>
fn = lambda scope: method(self.clone(parent=scope), *args, **kwargs)
File "/ist/users/pakkaponp/.conda/envs/jax/lib/python3.6/site-packages/flax/linen/module.py", line 221, in wrapped_module_method
return fun(self, *args, **kwargs)
File "/ist/ist-share/vision/pakkapon/nerf_proj/nerfies/nerfies/models.py", line 204, in __call__
metadata_encoded)
File "/ist/users/pakkaponp/.conda/envs/jax/lib/python3.6/site-packages/flax/linen/module.py", line 221, in wrapped_module_method
return fun(self, *args, **kwargs)
File "/ist/users/pakkaponp/.conda/envs/jax/lib/python3.6/site-packages/flax/linen/transforms.py", line 148, in wrapped_fn
ret = trafo_fn(get_module_scopes(self), *args, **kwargs)
File "/ist/users/pakkaponp/.conda/envs/jax/lib/python3.6/site-packages/flax/core/lift.py", line 180, in wrapper
*args)
File "/ist/users/pakkaponp/.conda/envs/jax/lib/python3.6/site-packages/flax/core/lift.py", line 379, in inner
return mapped(variable_groups, rng_groups, args)
File "/ist/users/pakkaponp/.conda/envs/jax/lib/python3.6/site-packages/jax/api.py", line 1226, in batched_fun
).call_wrapped(*args_flat)
File "/ist/users/pakkaponp/.conda/envs/jax/lib/python3.6/site-packages/jax/linear_util.py", line 166, in call_wrapped
ans = self.f(*args, **dict(self.params, **kwargs))
File "/ist/users/pakkaponp/.conda/envs/jax/lib/python3.6/site-packages/flax/core/lift.py", line 376, in mapped
y = fn(scope, *args)
File "/ist/users/pakkaponp/.conda/envs/jax/lib/python3.6/site-packages/flax/linen/transforms.py", line 141, in core_fn
res = fn(cloned, *args, **kwargs)
File "/ist/users/pakkaponp/.conda/envs/jax/lib/python3.6/site-packages/flax/linen/module.py", line 221, in wrapped_module_method
return fun(self, *args, **kwargs)
File "/ist/users/pakkaponp/.conda/envs/jax/lib/python3.6/site-packages/flax/linen/transforms.py", line 148, in wrapped_fn
ret = trafo_fn(get_module_scopes(self), *args, **kwargs)
File "/ist/users/pakkaponp/.conda/envs/jax/lib/python3.6/site-packages/flax/core/lift.py", line 180, in wrapper
*args)
File "/ist/users/pakkaponp/.conda/envs/jax/lib/python3.6/site-packages/flax/core/lift.py", line 379, in inner
return mapped(variable_groups, rng_groups, args)
File "/ist/users/pakkaponp/.conda/envs/jax/lib/python3.6/site-packages/jax/api.py", line 1226, in batched_fun
).call_wrapped(*args_flat)
File "/ist/users/pakkaponp/.conda/envs/jax/lib/python3.6/site-packages/jax/linear_util.py", line 166, in call_wrapped
ans = self.f(*args, **dict(self.params, **kwargs))
File "/ist/users/pakkaponp/.conda/envs/jax/lib/python3.6/site-packages/flax/core/lift.py", line 376, in mapped
y = fn(scope, *args)
File "/ist/users/pakkaponp/.conda/envs/jax/lib/python3.6/site-packages/flax/linen/transforms.py", line 141, in core_fn
res = fn(cloned, *args, **kwargs)
File "/ist/users/pakkaponp/.conda/envs/jax/lib/python3.6/site-packages/flax/linen/module.py", line 221, in wrapped_module_method
return fun(self, *args, **kwargs)
File "/ist/ist-share/vision/pakkapon/nerf_proj/nerfies/nerfies/warping.py", line 329, in __call__
warped_points = self.warp(points, metadata, alpha, metadata_encoded)
File "/ist/users/pakkaponp/.conda/envs/jax/lib/python3.6/site-packages/flax/linen/module.py", line 221, in wrapped_module_method
return fun(self, *args, **kwargs)
File "/ist/ist-share/vision/pakkapon/nerf_proj/nerfies/nerfies/warping.py", line 280, in warp
points_embed = self.points_encoder(points, alpha=alpha)
File "/ist/users/pakkaponp/.conda/envs/jax/lib/python3.6/site-packages/flax/linen/module.py", line 221, in wrapped_module_method
return fun(self, *args, **kwargs)
File "/ist/ist-share/vision/pakkapon/nerf_proj/nerfies/nerfies/modules.py", line 178, in __call__
window = self.cosine_easing_window(self.num_freqs, alpha)
File "/ist/ist-share/vision/pakkapon/nerf_proj/nerfies/nerfies/modules.py", line 201, in cosine_easing_window
x = jnp.clip(alpha - jnp.arange(num_freqs, dtype=jnp.float32), 0.0, 1.0)
File "/ist/users/pakkaponp/.conda/envs/jax/lib/python3.6/site-packages/jax/core.py", line 498, in __sub__
def __sub__(self, other): return self.aval._sub(self, other)
File "/ist/users/pakkaponp/.conda/envs/jax/lib/python3.6/site-packages/jax/_src/numpy/lax_numpy.py", line 5370, in deferring_binary_op
return binary_op(self, other)
File "/ist/users/pakkaponp/.conda/envs/jax/lib/python3.6/site-packages/jax/_src/numpy/lax_numpy.py", line 418, in <lambda>
fn = lambda x1, x2: lax_fn(*_promote_args(numpy_fn.__name__, x1, x2))
File "/ist/users/pakkaponp/.conda/envs/jax/lib/python3.6/site-packages/jax/_src/lax/lax.py", line 348, in sub
return sub_p.bind(x, y)
File "/ist/users/pakkaponp/.conda/envs/jax/lib/python3.6/site-packages/jax/core.py", line 259, in bind
out = top_trace.process_primitive(self, tracers, params)
File "/ist/users/pakkaponp/.conda/envs/jax/lib/python3.6/site-packages/jax/interpreters/partial_eval.py", line 1049, in process_primitive
out_avals = primitive.abstract_eval(*avals, **params)
File "/ist/users/pakkaponp/.conda/envs/jax/lib/python3.6/site-packages/jax/_src/lax/lax.py", line 2023, in standard_abstract_eval
return ShapedArray(shape_rule(*avals, **kwargs), dtype_rule(*avals, **kwargs),
File "/ist/users/pakkaponp/.conda/envs/jax/lib/python3.6/site-packages/jax/_src/lax/lax.py", line 2119, in _broadcasting_shape_rule
raise TypeError(msg.format(name, ', '.join(map(str, map(tuple, shapes)))))
TypeError: sub got incompatible shapes for broadcasting: (2,), (8,).
