The AI wave is sweeping in. Many people previously believed that “Prompt word engineer" will become an emerging type of work, and MindEye The advent ofThis position may no longer have value.
Previously, many people believed that the future of AI does not depend on whether a certain model is powerful, but on whether humans can use these AI models more efficiently to complete specific tasks.
This also gave birth to the concept of "prompt word engineer", who can understand AI better than ordinary people and can come up with more accurate prompt words so that AI can meet its required output.
and StabilityAI launched MindEye1 in July 2023, and recently launched MindEye2, which greatly reduced the value of "prompt word engineers". The model does not rely on specific prompt words, but is directly generated based on the user's brain waves. In other words, what the user's brain wants, the model will be able to generate in the future.
MindEye reconstructs and retrieves images directly from fMRI brain activity and can convert two-dimensional images into three-dimensional videos.
Functional magnetic resonance imaging (fMRI) is a neuroimaging technique that measures brain activity by detecting changes in blood flow. This technique is used to map brain function and evaluate potential treatments for neurological disorders.
MindEye is based on a dataset of brain activity recorded while participants viewed a series of static images in an MRI scanner. The research team trained the system to analyze these recordings and either retrieve the original image from a pool of candidates (retrieval) or generate a reconstruction of the image being viewed.
The researchers demonstrated that MindEye outperformed previous methods in image retrieval tasks, with an accuracy of more than 90% in identifying the original image from the candidate images. For reconstruction, MindEye uses a pre-trained generative model.
MindEye can be applied in various fields. In the medical field, its ability to reconstruct visual perception from brain activity can be used for diagnostic and assessment methods, especially in cases where patients have difficulty communicating. MindEye's real-time analysis potential is expected to improve the performance of brain-computer interfaces.
The team highlighted limitations related to data collection, including the lengthy scanning times required, and potential data noise due to participant movement or inattention.
Attached paper reference
-
Reconstructing the Mind's Eye: fMRI-to-Image with Contrastive Learning and Diffusion Priors
-
MindEye2: Shared-Subject Models Enable fMRI-To-Image With 1 Hour of Data
-
MindEye2: Shared-Subject Models Enable fMRI-To-Image With 1 Hour of Data