Source: Google AI, Editor: Xiao Qin, Jin Lei, Zhang Jia
【新智元导读】Today, Google, in collaboration with Howard · Hughes Medical Institute (HHMI) and the University of Cambridge, has released an intensive study of the fruit of the Drosophila brain — — automatically reconstructs the entire fruit fly brain. They use thousands of Google Cloud TPUs to rebuild the entire fruit fly brain up to 4 billion pixels. With a complete brain image, scientists are one step closer to understanding how the brain works.
do you know? Drosophila is recognized as one of the most thoroughly studied organisms. Up to now, eight Nobel Prizes have been awarded to research using fruit flies, which have driven the development of molecular biology, genetics and neuroscience.
Scientists have always dreamed of understanding how the nervous system works by drawing the structure of a complete brain neural network.
A major goal of recent research isDrosophila brain.
An important advantage of fruit flies is their size:Drosophila's brain is relatively small, with only 100,000 neurons, compared to 100 million neurons in the brain and 100 billion neurons in the human brain..
This makes the brain of the fruit fly easier to study as a complete circuit.
Today, Google has teamed up with Howard · Hughes Medical Institute (HHMI) and the University of Cambridge to publish a new in-depth study of the fruit fly brain research ——Automatically reconstruct the entire fruit fly brain.
Automatic reconstruction of Drosophila brain
This paper is entitled “Automatic Reconstruction of Continuous Slice Imaging of Drosophila Brain Using Flood-Filling Network and Local Adjustments”:
A total of 16 researchers from Google, Howard · Hughes Medical Research Institute (HHMI) Janelia Research Park and Cambridge University participated in the study. The first author, Peter H. Li, is a Google research scientist whose main research interests include general Science, machine intelligence, machine perception.
Peter H. Li
They also provide a complete picture of the Drosophila brain, which anyone can download and view, or browse online using interactive tools. They developed a 3D interactive interface called Neuroglancer.
Neuroglancer demo video please gooriginalWatch.
This is not the first time that the Drosophila brain has been completely drawn. In January of this year, Science magazine covered the cover story and introduced MIT and Howard · Hughes Medical Research Institute (HHMI) scientists successfully imaged the complete brain of Drosophila. And the resolution has reached the nanometer level. But that was still an artificial method, using two of the most advanced microscope techniques.
For decades, neuroscientists have been dreaming of drawing a complete map of the brain's neural network, but for a human brain with 100 billion neural networks, the amount of data that needs to be processed is unimaginable. If you can automatically rebuild the Drosophila brain, it may be a step closer to automatically drawing the human brain.
This is not the first time Peter H. Li's team has attempted to map brain neurons using the AI method. They studied on smaller datasets in 2016 and 2018, respectively, as shown in the lower right corner of the image below.
The lower right corner of a 40 trillion pixel Drosophila brain is the smaller data set analyzed by Google AI in 2016 and 2018, respectively.
In 2018,Google has collaborated with the Max Planck Institute for Neurobiology in Germany to develop a deep learning-based system that automatically maps neurons in the brain.. They reconstructed a scanned image of the brain of a 1 million cubic micron zebra finch.
Researchers say that due to the high resolution of imaging,Even with only one cubic millimeter of brain tissue, it can generate more than 1000TB of data.. Therefore, this time rebuilding the brain of the whole fruit fly, I think the amount of data is huge.
Used to process data, is Google’sCloud TPUAnd it is thousands!
Google AI director Jeff Dean also sighed on Twitter:
TPU will really fly! GoogleAI scientists use TPU to help rebuild the neural connections throughout the fruit fly brain!
Below, the new wisdom brings a detailed interpretation of this research:
One, 40 trillion pixel fruit fly brain, automatic reconstruction!
In the course of the experiment, the main data set used was FAFB, which is an abbreviation for “full adult fly brain” (see the end of the relevant dataset for information).
Researchers on this data set will brain the fruit flyCut into thousands of ultra-thin slices of 40 nanometersEach slice was then imaged using a transmission electron microscope, which produced more than 40 trillion pixels of brain images. And these 2D images are integrated into a coherent 3D Drosophila brain image.
Next, the researchers used thousands of cloud TPUs and applied them.Flood-Filling Network (FFN)In order to automatically track each neuron in the Drosophila brain.
Dense segmentation of the entire Drosophila brain through FFN
A in the above figure is a 3D rendered FAFB data set smoothed tissue mask. Any coronal slice (dataset XY plane) shows the entire internal FAFB-FFN1 segmentation. B-E shows the effect of increasing the zoom ratio.
Automated neuron reconstruction and manual neuron tracking for verification
Although the overall performance of this algorithm is not bad, when the alignment is not perfect (the image content in the continuous slice is unstable) or occasionally due to the loss of multiple consecutive slices during the imaging process, the performance will be degraded.
In order to make up for this problem, the researchers willCombine FFN with two new programs.
First, the consistency between the slices of each region in the 3D image is estimated, and then the content in the image is locally stabilized while the FFN tracks each neuron.
Second, the researchers used SECGAN to calculate missing slices in the image volume, and when using SECGAN, the researchers found that FFN could more reliably track the location of multiple missing slices.
Local Realignment (LR)
Irregular section replacement
Segmentation of the overall FAFB-FFN1
Segmentation-assisted neuron tracking
Second, interactive visualization of Drosophila brain and Neuroglancer
Visualization is both important and difficult when dealing with 3D images that contain trillions of pixels and complex shapes. Inspired by Google's history of developing new visualization technologies, researchers have designed a new, scalable and powerful tool that anyone with a Web browser that supports WebGL can access.
The result is Neuroglancer, an open source project on github that looks at petabyte-level 3D volume and supports many advanced features such as arbitrary-axis cross-sectional reslicing, multi-resolution meshing, and Integrate with Python to develop the power of custom analytic workflows. The tool has been widely used by collaborators, including Allen Brain Science Institute, Harvard University, HHMI, Max Planck Institute, MIT, and Princeton University.
Third, future work
Google said that HHMI and Cambridge collaborators have begun to use this reconstruction to accelerate their research on Drosophila brain learning, memory and perception. However, since the establishment of the connection group requires the identification of synapses, the above result is not a true connectome. They are working closely with the FlyEM team at Janelia Research Campus to create a highly validated and detailed connectome for the Drosophila brain using images obtained with the FIB-SEM & rdquo; technology.
FAFB: v14 arranged whole adult fruit fly brain
FAFB CLAHE: FAFB and CLAHE contrast normalization
CREMI: CREMI Challenge Training Data Based on Early FAFB Arrangement