IMAGING SCIENCE AND TECHNOLOGY FOR MATERIAL APPEARANCE
REALNESS IN DIGITAL IMAGES
-Understanding how the human brain interprets images could underpin the future of internet shopping, cosmetics, digital fabrication and even non-invasive medicine.
Research Keywords: Texture Engineering, Image Sensing, Visual Psychology
© iStock.com/wacomkaWhen internet purchases arrive and seem nothing like their picture, it can be a disappointing experience for shoppers. So, while big e-commerce players are increasingly investing in technologies to help personalize products or shop via phone or virtual reality headset, a significant stumbling block is rendering realistic, digitally produced images.
“To achieve a realistic rendering of something, we also need to deeply consider human perception,” explains Keita Hirai, a member of the Imaging Science and Technology for Material Appearance group at Chiba University’s Graduate School of Engineering. How we perceive a material, he says, can also be linked to cognition, not just physical phenomena.
To examine the complex effect of the mind’s eye, Chiba University have drawn from a diverse range of researchers, including engineers, psychologists, and medical and cognitive scientists.
Recently, Hirai's team developed a new system for texture evaluation, and through participant feedback have devised a set of algorithms that can produce more realistic digital representations of patinas and colours overlaid onto textured fabrics in digital images. The aim, he says, is to one day digitally simulate photo-real fabric appearance via a smart device. Hirai points out that this will not only be helpful to internet shoppers, but that these algorithms could also find applications in rapidly advancing digital fabrication techniques, such as printing digital patinas onto fabrics or objects, and manufacturing representative digitally-designed 3D products. In fact, the group aims to contribute to industries as diverse as cosmetics and medicine.
FILTERED THROUGH THE BRAIN
Among the work done by the diverse group is a study examining texture. Viewers, suggest the researchers, often see the broad design of patinas printed on fabric, but some textures contribute to a blurring out of finer details. In contrast, high-resolution digital images with very accurate representations of the same material seem unnaturally sharp. To explore this, the researchers developed different ‘blurriness’ filters linked to material qualities and fine-tuned them by asking participants in their experiment to rate the naturalness of the images.
Another group found that our perception of colour is affected by our understanding of its dimensionality. The same viewer looking at the same image saw colours differently depending on whether they interpreted the image as two dimensional or three dimensional. Viewing images with one eye or both, and whether the image or the viewer’s head was moving, also affected how natural the image seemed. This work, says Hirai, is important for better smart device production. “A more informed colour management system, for example, may help realize more accurate colour reproduction between different devices: displays, cameras and printers,” he explains.
Hirai says the group's work will influence the most cutting-edge imaging technology, such as the omni-directional 3D camera systems and virtual reality headsets coming from technology giants such as Sony, Panasonic, Canon and Nikon. While he also mentions cosmetics and automotive painting, one team have already had some remarkable success in one crucial, but quite unique area — medical diagnostics.
These researchers have carefully studied the optical properties of haemoglobin in skin using images taken on a simple DSLR camera. Incredibly, they found they could measure both the heart rate and its variability with 99 percent accuracy. This opens the door to everything from non-invasive medical diagnosis to a better understanding of physical exertion using a simple smartphone app.
（CHIBA RESEARCH 2020）Members
Principal Investigator
| Name | Title, Affiliation | Research Themes |
|---|---|---|
| TSUMURA Norimichi | Associate Professor, Graduate School of Engineering | Color Dynamics, Pattern Recognition |
Co-Investigatior
| Name | Title, Affiliation | Research Themes |
|---|---|---|
| HORIUCHI Takahiko | Professor, Graduate School of Engineering | Texture Image Engineering, Visual Engineering, Color Engineering |
| HIRAI Keita | Associate Professor, Graduate School of Engineering | Color Information Processing |
| MIZOKAMI Yoko | Professor, Graduate School of Engineering | Visual Information Processing |
| IMAIZUMI Shoko | Associate Professor, Graduate School of Engineering | Information Security |
| KOHRI Michinari | Associate Professor, Graduate School of Engineering | Materials Engineering |
| YATA Noriko | Assistant Professor, Graduate School of Engineering | Evolutionary Computation, Neural Network |
| MATSUKA Toshihiko | Professor, Graduate School of Humanities and Studies on Public Affairs | Cognitive Modeling |
| YOSHIMURA Kensuke | Professor,Chiba University Hospital | Medical management,Medical informatics,Neuropsychiatry |
| TOKUNAGA Rumi | Associate Professor,College of Liberal Arts and Sciences | Color engineering,Visual information processing |
| AMEMIYA Ayumi | Assistant Professor,Graduate School of NursingGraduate School of NursingGraduate School of Nursing | Nursing science and engineering |
| NAKAMURA Kazuki | Associate Professor, Graduate School of Engineering | Optical functional material,Optical property measurement |
| KOTSUKI Shunji | Associate Professor, Center for Environmental Remote Sensing | AI,Big data analysis,Numerical modeling simulation |
| INOUE Shinichi | Associate Professor, IGPR | Optical, Measurement system, Modeling | HANAZATO Masamichi | Associate Professor, Center for Preventive Medical Science | Healthy city/architecture,Public health | SHIINA Tatsuo | Associate Professor, Graduate School of Engineering | Optical measurement,Optical data analysis |
| KOMURO Nobuyoshi | Associate Professor, Institute of Management and Information Tschnologies | Wireless communication,Wireless Sensor Network,Environmental sensing |
| SAKURADA Tomoya | Associate Professor,Graduate School of Pharmaceutical Science | Social Pharmacy,Regulatory Science |
| SATO Hiromi | Assistant Professor,Graduate School of Engineering | Visual psychology,Emotion mechanism |
Research report（2017〜2019）
The achievement of this promotion research from pure academic point of view is obvious from the doubling of the 39 journal papers by the 5 core members in the 3 years before the program started, to 87 journal papers during the 3 years of the program period. Research activities had progressed sufficiently against the original research plan. For example, during the program period of three years, in addition to addressing issues such as “realization of embedding SHITSUKAN information in digital images” described in the research plan, we promoted joint research with a company that handled 3D printers. Then, we developed a technology to generate human skin with real transparent characteristics using a 3D printer. Furthermore, we were able to accelerate the interdisciplinary approach by adding experts in behavioral science, information engineering, and materials engineering to the core members and experts in medical engineering to the collaborating researchers; this was beyond the scope of the originally planned research field of imaging technology.
Furthermore, we actively built domestic and overseas research networks during the program period. We started with 12 corporate researchers, mainly graduates of our university, as collaborative researchers. During the program period, we appointed 30 corporate researchers as advisors to promote information sharing related to SHITSUKAN imaging research and held open labs and discussion meetings. Additionally, we invited two overseas collaborative researchers to Chiba University for detailed discussion at an international workshop on material appearance (a part of SHITSUKAN) sponsored by this program. Furthermore, the network was expanded by seeking cooperation from the SHITSUKAN projects that are being implemented in other countries. Particularly, the laboratories in this program were included in several international graduate school education programs, and in addition to research, expanded the international network of education programs. Moreover, during the program period, students and researchers underwent exchange programs, mainly in Asia and Europe.
During the program period, we actively built academia and networks with industry. As a result, collaborative research with several companies was actively promoted by individual core members— the results in terms of social implementation and innovation creation exceeded expectations. For example, we developed products with multiple companies for the contactless emotion monitoring technology developed in this program. Additionally, commercialization was considered by several companies, including the cosmetics and printing industries. From the viewpoint of innovation creation, in the field of emotional engineering, the responsible person participated in the COI program center: “Center of KANSEI Innovation Nurturing Mental Wealth,” and the innovation creation activities progressed.
Another noteworthy outcome of this program was our focus on training young human resources. Through the ongoing Asia Student Workshop, we have accepted students from Asian countries and conducted texture-related education. In addition, it was decided that, as part of a newly developed graduate education program, the laboratories related to this program would be included in the education project of material appearance for master's degree students; this has been implemented mainly by Norway since 2019. Since 2020, several master's degree students have planned short- and middle-term studies abroad through the SHITSUKAN research. Furthermore, it has been decided to include the laboratories related to this program in the international Master of Science in Imaging and Light in Extended Reality program, which is being implemented mainly by Finland since 2020. This has triggered discussions on the development of a future double degree system.