Predicted image becomes white

[saneatsu]

Problem

Currently, I am creating a model using images in my house for learning images.
The original model has not changed.
When predicting images, it becomes white like the image below.
I changed the loss function L6 (loss_tv) from 0.1 to 1, but it was ineffective.

What I changed

libs/pconv_model.py in loss_total func

return l1 + 6*l2 + 0.05*l3 + 120*(l4+l5) + 1.0*l6

How to predict

Input image examples

mask image

In order to avoid recognizing it as a mask when there is a pure white color in the original image, the whole is grayed out.

original color image

Code

model_rootpath = '/mnt/PConv-Keras/'
model = PConvUnet(weight_filepath='data/model/')
model.load(
    '{}/01/weight/13_weights_2019-03-09-00-20-20.h5'.format(model_rootpath),
    train_bn=False,
    lr=0.00005
)

SAVE_IMG_ROOTPATH = '/mnt/PConv-Keras/predicted_imgs'
CASE_NAME         = 'CASE-01'
GPU_SIZE          = 'GPU-1'
BATCH_SIZE        = 'Batch-07'
EPOCH             = 'Epoch-13'
LEARNING_TIME     = 'Time-200h'

# Create dir
#   ex) CASE-01_GPU-4_Batch-27_Epoch-37_Time-46h
save_filename = ('_').join((CASE_NAME, GPU_SIZE, BATCH_SIZE, EPOCH, LEARNING_TIME))
dir_name = datetime.now().strftime('%Y%m%d_%H%M%S')
save_dir_path = SAVE_IMG_ROOTPATH + '/' + dir_name
os.makedirs(save_dir_path)

start = time.time()
for i in range(6):
    # Load masked image and Create mask image(black and white)
    masked_img = cv2.imread('{0}/input_imgs/img_{1:02d}_mask.jpg'.format(SAVE_IMG_ROOTPATH, i))    
    masked_img = cv2.resize(masked_img, (cst.IMG_WIDTH, cst.IMG_HEIGHT))
    masked_img = cv2.cvtColor(masked_img, cv2.COLOR_BGR2RGB)
    
    # Create mask image
    mask = np.concatenate([((np.sum(masked_img, axis = -1) > 200 * 3) * 255)[..., np.newaxis]] * 3, axis = -1)
    mask_inv_img = 255 - mask

    # Load original color image
    ori_img = cv2.imread('{0}/input_imgs/img_{1:02d}_ori.jpg'.format(SAVE_IMG_ROOTPATH, i))
    ori_img = cv2.resize(ori_img, (cst.IMG_WIDTH, cst.IMG_HEIGHT))
    ori_img = cv2.cvtColor(ori_img, cv2.COLOR_BGR2RGB)

    # Resize for prediction
    model_input_img_tensor = ori_img[np.newaxis, ...]/255.
    model_input_mask_tensor = mask_inv_img[np.newaxis, ...]/255
    
    # Prediction
    model_output = model.predict([model_input_img_tensor, model_input_mask_tensor])
    save_img = Image.fromarray(np.uint8((model_output[0,:,:,:] * 1.)*255))
    save_img.save('{0}/{1}_{2:02d}.jpg'.format(save_dir_path, save_filename, i))

Predicted image

[MathiasGruber]

I've deployed my model on www.fixmyphoto.ai. With it, I get the following:

[yosuke-yasuda]

I got the same problem. I didn't want to change image pixels out of the mask, so I created an image with predicted values in the mask and original values out of the mask. But the predicted values on the edge of the mask area is different from original image and got a similar one with the @saneatsu 's predicted image. Is it possible to get natural image using original pixels out of the mask?

[ttxxr]

Problem

Currently, I am creating a model using images in my house for learning images.
The original model has not changed.
When predicting images, it becomes white like the image below.
I changed the loss function L6 (loss_tv) from 0.1 to 1, but it was ineffective.

What I changed

libs/pconv_model.py in loss_total func

return l1 + 6*l2 + 0.05*l3 + 120*(l4+l5) + 1.0*l6

How to predict

Input image examples

mask image

In order to avoid recognizing it as a mask when there is a pure white color in the original image, the whole is grayed out.

original color image

Code

model_rootpath = '/mnt/PConv-Keras/'
model = PConvUnet(weight_filepath='data/model/')
model.load(
    '{}/01/weight/13_weights_2019-03-09-00-20-20.h5'.format(model_rootpath),
    train_bn=False,
    lr=0.00005
)

SAVE_IMG_ROOTPATH = '/mnt/PConv-Keras/predicted_imgs'
CASE_NAME         = 'CASE-01'
GPU_SIZE          = 'GPU-1'
BATCH_SIZE        = 'Batch-07'
EPOCH             = 'Epoch-13'
LEARNING_TIME     = 'Time-200h'

# Create dir
#   ex) CASE-01_GPU-4_Batch-27_Epoch-37_Time-46h
save_filename = ('_').join((CASE_NAME, GPU_SIZE, BATCH_SIZE, EPOCH, LEARNING_TIME))
dir_name = datetime.now().strftime('%Y%m%d_%H%M%S')
save_dir_path = SAVE_IMG_ROOTPATH + '/' + dir_name
os.makedirs(save_dir_path)

start = time.time()
for i in range(6):
    # Load masked image and Create mask image(black and white)
    masked_img = cv2.imread('{0}/input_imgs/img_{1:02d}_mask.jpg'.format(SAVE_IMG_ROOTPATH, i))    
    masked_img = cv2.resize(masked_img, (cst.IMG_WIDTH, cst.IMG_HEIGHT))
    masked_img = cv2.cvtColor(masked_img, cv2.COLOR_BGR2RGB)
    
    # Create mask image
    mask = np.concatenate([((np.sum(masked_img, axis = -1) > 200 * 3) * 255)[..., np.newaxis]] * 3, axis = -1)
    mask_inv_img = 255 - mask

    # Load original color image
    ori_img = cv2.imread('{0}/input_imgs/img_{1:02d}_ori.jpg'.format(SAVE_IMG_ROOTPATH, i))
    ori_img = cv2.resize(ori_img, (cst.IMG_WIDTH, cst.IMG_HEIGHT))
    ori_img = cv2.cvtColor(ori_img, cv2.COLOR_BGR2RGB)

    # Resize for prediction
    model_input_img_tensor = ori_img[np.newaxis, ...]/255.
    model_input_mask_tensor = mask_inv_img[np.newaxis, ...]/255
    
    # Prediction
    model_output = model.predict([model_input_img_tensor, model_input_mask_tensor])
    save_img = Image.fromarray(np.uint8((model_output[0,:,:,:] * 1.)*255))
    save_img.save('{0}/{1}_{2:02d}.jpg'.format(save_dir_path, save_filename, i))

Predicted image

have u solved the problem? can u share your model , please?

[flyonok]

Problem

Currently, I am creating a model using images in my house for learning images.
The original model has not changed.
When predicting images, it becomes white like the image below.
I changed the loss function L6 (loss_tv) from 0.1 to 1, but it was ineffective.

What I changed

libs/pconv_model.py in loss_total func

return l1 + 6*l2 + 0.05*l3 + 120*(l4+l5) + 1.0*l6

How to predict

Input image examples

mask image

In order to avoid recognizing it as a mask when there is a pure white color in the original image, the whole is grayed out.

original color image

Code

model_rootpath = '/mnt/PConv-Keras/'
model = PConvUnet(weight_filepath='data/model/')
model.load(
    '{}/01/weight/13_weights_2019-03-09-00-20-20.h5'.format(model_rootpath),
    train_bn=False,
    lr=0.00005
)

SAVE_IMG_ROOTPATH = '/mnt/PConv-Keras/predicted_imgs'
CASE_NAME         = 'CASE-01'
GPU_SIZE          = 'GPU-1'
BATCH_SIZE        = 'Batch-07'
EPOCH             = 'Epoch-13'
LEARNING_TIME     = 'Time-200h'

# Create dir
#   ex) CASE-01_GPU-4_Batch-27_Epoch-37_Time-46h
save_filename = ('_').join((CASE_NAME, GPU_SIZE, BATCH_SIZE, EPOCH, LEARNING_TIME))
dir_name = datetime.now().strftime('%Y%m%d_%H%M%S')
save_dir_path = SAVE_IMG_ROOTPATH + '/' + dir_name
os.makedirs(save_dir_path)

start = time.time()
for i in range(6):
    # Load masked image and Create mask image(black and white)
    masked_img = cv2.imread('{0}/input_imgs/img_{1:02d}_mask.jpg'.format(SAVE_IMG_ROOTPATH, i))    
    masked_img = cv2.resize(masked_img, (cst.IMG_WIDTH, cst.IMG_HEIGHT))
    masked_img = cv2.cvtColor(masked_img, cv2.COLOR_BGR2RGB)
    
    # Create mask image
    mask = np.concatenate([((np.sum(masked_img, axis = -1) > 200 * 3) * 255)[..., np.newaxis]] * 3, axis = -1)
    mask_inv_img = 255 - mask

    # Load original color image
    ori_img = cv2.imread('{0}/input_imgs/img_{1:02d}_ori.jpg'.format(SAVE_IMG_ROOTPATH, i))
    ori_img = cv2.resize(ori_img, (cst.IMG_WIDTH, cst.IMG_HEIGHT))
    ori_img = cv2.cvtColor(ori_img, cv2.COLOR_BGR2RGB)

    # Resize for prediction
    model_input_img_tensor = ori_img[np.newaxis, ...]/255.
    model_input_mask_tensor = mask_inv_img[np.newaxis, ...]/255
    
    # Prediction
    model_output = model.predict([model_input_img_tensor, model_input_mask_tensor])
    save_img = Image.fromarray(np.uint8((model_output[0,:,:,:] * 1.)*255))
    save_img.save('{0}/{1}_{2:02d}.jpg'.format(save_dir_path, save_filename, i))

Predicted image

@saneatsu @MathiasGruber Has your problem been solved?

[SunnyPann]

Are you training on your own training set?Where can I download the training model?thank you