This study proposed an innovative technique to measure the fish color in situ based on machine vision to avoid potential effects on fish health that might result from conventional measurement with a colorimeter. The research goal was achieved by developing a color conversion model trained with images of a color reference target in the ambient environment and clay samples in the aquarium representing the target fish. Brightness adjustment was effectively applied to these images to minimize uneven illumination in both environments. The generalization ability of the model was ensured with regularized regression. Consequently, an optimal cubic ridge model was obtained with appreciably low training and testing mean color errors of 1.17 ± 0.64 and 1.10 ± 0.50 CIEDE2000 color difference units, respectively. Preliminary results of measuring the clownfish color showed that the proposed system had much potential for in situ and non-invasive visualizing of the color distribution of small ornamental fish with color non-uniformity.