error in creating tensorflow dataset

[buffyhridoy]

trafficstars

def load_data(point_cloud_batch, label_cloud_batch): point_cloud_batch.set_shape([NUM_SAMPLE_POINTS, 7]) #NUM_SAMPLE_POINTS=1024 label_cloud_batch.set_shape([NUM_SAMPLE_POINTS, 9]) return point_cloud_batch, label_cloud_batch

def augment(point_cloud_batch, label_cloud_batch): noise = tf.random.uniform( tf.shape(label_cloud_batch), -0.005, 0.005, dtype=tf.float64 ) point_cloud_batch += noise[:, :, :3] return point_cloud_batch, label_cloud_batch

def generate_dataset(point_clouds, label_clouds, is_training=True): dataset = tf.data.Dataset.from_tensor_slices((point_clouds, label_clouds)) dataset = dataset.shuffle(BATCH_SIZE * 100) if is_training else dataset dataset = dataset.map(load_data, num_parallel_calls=tf.data.AUTOTUNE) dataset = dataset.batch(batch_size=BATCH_SIZE) dataset = ( dataset.map(augment, num_parallel_calls=tf.data.AUTOTUNE) if is_training else dataset ) return dataset

split_index = int(len(point_clouds) * (1 - VAL_SPLIT)) train_point_clouds = point_clouds[:split_index] train_label_cloud = point_cloud_labels[:split_index] total_training_examples = len(train_point_clouds)

val_point_clouds = point_clouds[split_index:] val_label_cloud = point_cloud_labels[split_index:]

print("Num train point clouds:", len(train_point_clouds)) print("Num train point cloud labels:", len(train_label_cloud)) print("Num val point clouds:", len(val_point_clouds)) print("Num val point cloud labels:", len(val_label_cloud))

train_dataset = generate_dataset(train_point_clouds, train_label_cloud) val_dataset = generate_dataset(val_point_clouds, val_label_cloud, is_training=False)

print("Train Dataset:", train_dataset) print("Validation Dataset:", val_dataset)

############################################################################################## the result should be:

Num train point clouds: 819 Num train point cloud labels: 819 Num val point clouds: 205 Num val point cloud labels: 205

Train Dataset: <ParallelMapDataset shapes: ((None, 1024, 7), (None, 1024, 9)), types: (tf.float64, tf.float32)> Validation Dataset: <BatchDataset shapes: ((None, 1024, 7), (None, 1024, 9)), types: (tf.float64, tf.float32)> ############################################################################################### but we are getting error in line 2:

Num train point clouds: 819 Num train point cloud labels: 819 Num val point clouds: 205 Num val point cloud labels: 205

ValueError Traceback (most recent call last) in () 39 print("Num val point cloud labels:", len(val_label_cloud)) 40 ---> 41 train_dataset = generate_dataset(train_point_clouds, train_label_cloud) 42 val_dataset = generate_dataset(val_point_clouds, val_label_cloud, is_training=False) 43

11 frames /usr/local/lib/python3.7/dist-packages/tensorflow/python/autograph/impl/api.py in wrapper(*args, **kwargs) 697 except Exception as e: # pylint:disable=broad-except 698 if hasattr(e, 'ag_error_metadata'): --> 699 raise e.ag_error_metadata.to_exception(e) 700 else: 701 raise

ValueError: in user code:

File "<ipython-input-60-dd40329ada93>", line 2, in load_data  *
    point_cloud_batch.set_shape([NUM_SAMPLE_POINTS, 7])

ValueError: Shapes must be equal rank, but are 1 and 2

[SuryanarayanaY]

@buffyhridoy ,

I tried to replicate but it seems the dataset URL having issues as mentioned below.

Exception: URL fetch failure on https://git.io/JiY4i: 502 -- Bad Gateway

CC'ing the authors @sayakpaul , @soumik12345 , Could you please update the URL ?

Thanks!

[soumik12345]

Hi @SuryanarayanaY @buffyhridoy You can use this link: https://github.com/soumik12345/point-cloud-segmentation/releases/download/v0.1/shapenet.zip

[SuryanarayanaY]

@soumik12345 ,

Thanks for the link. I will check it and if working I am willing to raise a PR to fix this if you are fine with that. Thanks!