2d and 3d joints positions

[Marlod390]

I tried to use the smpl parameter to calculate the position of 3d joints in order to compare with the data under the 3d_joints key, but found that the two data are different. At the same time, when I visualized the 2d joints on the original image, I also found that their positions did not match the actual ones. So I want to know how the 3d joints and 2d joints are calculated. Thanks in advance

[geopavlakos]

You can follow the code in the forward_step to see how we calculate the 3D joints and project them to the image.
In what way were the 3D and 2D joints different from the ones you calculated?

[Marlod390]

Thank for your reply. This is the code i use to visualize the first 25 predicted 2d joints on the original frame.
`import joblib
import cv2
import matplotlib.pyplot as plt
import numpy as np

data = joblib.load('/home/markusc/Videos/panda_test/video/output2/webvid/medium/3/4dhumans/3_4dhumans.pkl')

first_key = next(iter(data.keys()))
joints_2d_1 = data[first_key]['2d_joints'][0][:252]
joints_2d_2 = data[first_key]['2d_joints'][1][:252]

image_path = '/home/markusc/Videos/panda_test/video/3/json_files/multiple/3/0_pose.png'
image = cv2.imread(image_path)
image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)

img_height, img_width, _ = image.shape

joints_2d_1 = joints_2d_1.reshape(-1, 2)
joints_2d_2 = joints_2d_2.reshape(-1, 2)

joints_2d_1[:, 0] *= img_width
joints_2d_1[:, 1] *= img_height
joints_2d_2[:, 0] *= img_width
joints_2d_2[:, 1] *= img_height

print("Joints 2D 1 positions:")
print(joints_2d_1)

print("Joints 2D 2 positions:")
print(joints_2d_2)

plt.figure(figsize=(10, 10))
plt.imshow(image)

for idx, joint in enumerate(joints_2d_1):
if idx == 0:
plt.scatter(joint[0], joint[1], s=100, c='yellow', marker='o')
else:
plt.scatter(joint[0], joint[1], s=30, c='red', marker='o')
plt.text(joint[0], joint[1], str(idx), fontsize=12, color='red')

for idx, joint in enumerate(joints_2d_2):
if idx == 0:
plt.scatter(joint[0], joint[1], s=100, c='green', marker='o')
else:
plt.scatter(joint[0], joint[1], s=30, c='blue', marker='o')
plt.text(joint[0], joint[1], str(idx), fontsize=12, color='blue')

plt.title('2D Joints on Image - First Frame')
plt.axis('off')
plt.show()`

And the result is as follow

[Justinitaa]

@Marlod390 Hello, I would like to ask, for example, the following is the 3d_joints part of the information I extracted in the .pkl file, there are 45 lines in total, the last line looks like the origin, do you know how each line of information corresponds to each of the 44 joints? Do you know how each line of information corresponds to each of the 44 joints? For example, which joint is the first line of data?

[Justinitaa]

@Marlod390