|
1 | | -# DeepCell Label: A Web Application for Data Curation |
| 1 | +# DeepCell Label: Cloud-Based Labeling for Single-Cell Analysis |
2 | 2 |
|
3 | 3 | [](https://github.com/vanvalenlab/deepcell-label/actions) |
4 | 4 | [](https://coveralls.io/github/vanvalenlab/deepcell-label?branch=master) |
| 5 | +[](https://github.com/vanvalenlab/deepcell-label/blob/master/LICENSE) |
5 | 6 |
|
6 | | -DeepCell Label is a web-based data annotation tool based on Flask and can be run locally or deployed to the cloud (e.g. AWS Elastic Beanstalk). |
| 7 | +DeepCell Label is a web-based tool to visualize and label cells in biological images, and is tailored for cell segmentation. Label works with multiplexed images, 3D image stacks, and time-lapse movies. The tool can label images from scratch, or correct labels from model predictions or existing datasets. |
7 | 8 |
|
8 | | -For an up-to-date list of features, see the [Controls](#Controls) section. |
| 9 | +DeepCell Label distributes images and labeling tools through a browser, providing easy access to crowdsource data labeling and to review, correct, and curate labels as a domain expert. |
9 | 10 |
|
10 | | -## DeepCell Label for Developers |
| 11 | +DeepCell Label is built with [React](https://reactjs.org/), [XState](https://xstate.js.org/docs/), and [Flask](https://flask.palletsprojects.com/en/2.0.x/) and [runs locally](#local-use) or [on the cloud](#cloud-deployment) (e.g. with [AWS Elastic Beanstalk](https://aws.amazon.com/elasticbeanstalk/)). |
11 | 12 |
|
12 | | -### Install Dependencies |
| 13 | +Visit [label.deepcell.org](https://label.deepcell.org) and create a project from an example file or upload a .tiff, .png, or .npz. Detailed instructions on how to use the app are in the dropdown instructions tab when working in DeepCell Label. |
13 | 14 |
|
14 | | -Using a virtual environment to install dependencies is recommended. |
| 15 | +## Local use |
15 | 16 |
|
16 | | -```bash |
17 | | -pip install -r requirements.txt |
18 | | -``` |
| 17 | +To run DeepCell Label locally, you will set up the client and the server from your computer. |
19 | 18 |
|
20 | | -### Run browser application in development mode |
| 19 | +To begin, clone the repository with the following command: |
21 | 20 |
|
22 | | -```bash |
23 | | -python3 application.py |
24 | 21 | ``` |
25 | | - |
26 | | -### Use docker-compose for local development |
27 | | - |
28 | | -Add your AWS credentials to `docker-compose.yaml` or set them via the environment. |
29 | | - |
30 | | -From the root directory, run: |
31 | | - |
32 | | -```bash |
33 | | -sudo docker-compose up --build -d |
| 22 | +git clone https://github.com/vanvalenlab/deepcell-label.git |
| 23 | +cd deepcell-label |
34 | 24 | ``` |
35 | 25 |
|
36 | | -Wait a minute for the database to finish setting up before running: |
37 | | - |
38 | | -```bash |
39 | | -sudo docker-compose restart app |
40 | | -``` |
| 26 | +### Set up Flask server |
41 | 27 |
|
42 | | -You can now go to 0.0.0.0:5000 in a browser window to access the local version of the tool. |
| 28 | +The backend will store and update project data. |
43 | 29 |
|
44 | | -To interact with the local mysql database: |
| 30 | +It's recommended to set up a virtual environment with [conda](https://conda.io/projects/conda/en/latest/user-guide/tasks/manage-environments.html) or [venv](https://docs.python.org/3/library/venv.html). |
| 31 | +Install the Python dependencies with: |
45 | 32 |
|
46 | | -```bash |
47 | | -sudo docker exec -it browser_db_1 bash |
48 | | -mysql -p |
49 | 33 | ``` |
50 | | - |
51 | | -When finished: |
52 | | - |
53 | | -```bash |
54 | | -sudo docker-compose down |
| 34 | +pip install -r requirements.txt |
55 | 35 | ``` |
56 | 36 |
|
57 | | -(optional) |
| 37 | +Then, start the server with: |
58 | 38 |
|
59 | | -```bash |
60 | | -sudo docker system prune --volumes |
| 39 | +``` |
| 40 | +flask run |
61 | 41 | ``` |
62 | 42 |
|
63 | | -## Structure of Browser Version |
64 | | - |
65 | | -Flask is used as an HTTP server that serves the frames as pngs and metadata as JSON. The .js files in the `deepcell_label/template` folder are what makes the requests to the Flask server. |
66 | | - |
67 | | -Python Flask was used as a web application framework for constructing DeepCell Label. The Flask framework helps serves as the router that maps the specific URL with the associated function that is intended to perform some task. Specifically, the application.route decorator binds the URL rule to the function below it. Thus, if user performs actions that cause a change in the underlying data, the side-serving .js file will request to visits a specific URL, and the output of the function below the decorator will be rendered in the browser. |
68 | | - |
69 | | -Functions depend on Python libraries -- including NumPy, Matplotlib, and scikit-image – to change the data within files. After the desired change has been made to the lineage information or mask annotation, the Flask app routing will update the interface to reflect the alterations with support from side-serving JavaScript scripts. |
70 | | - |
71 | | -The final Flask application has been deployed to an AWS Elastic Beanstalk environment as a RESTful web service. A stable demo of the browser application can be accessed at label.deepcell.org. To deploy this application to AWS EB, an AWS RDS MySQL database must be set up and configured to handle data storage for application use. (Add database credentials to the .env configuration file.) Once a database is appropriately configured, the application can easily be launched by using the AWS EB command line tool or web interface. The .ebextensions folder will configure the web service to use the appropriate Flask application (eb_application.py, which uses a MySQL database instead of SQLite). |
72 | | - |
73 | | -DeepCell Label can also be run locally using a SQLite database (this is the default behavior). When we start using DeepCell Label, DeepCell Label creates a project with a unique 12 character ID, and stores it locally in `/tmp/deepcell_label.db`. Whenever we edit the project, DeepCell Label updates the object in the database. Running application.py creates the database if it does not already exist. |
74 | | - |
75 | | -## Layout |
76 | | - |
77 | | -Label shows an Info Table on the left with project details and an interactive Canvas on the right that shows images and labels. |
78 | | - |
79 | | -### Infopane |
80 | | - |
81 | | -The infopane is a table with information about the project, |
82 | | - |
83 | | -The first three rows in the table tell us the which slice in an image stack we are currently viewing |
84 | | - |
85 | | -- **frame** tells which slice of an image stack we are viewing |
86 | | - - Frames move through space (like a 3D vertical image stack) or |
87 | | - through time (like a timelapse) |
88 | | -- **channel** tells us which raw image channel we are viewing |
89 | | - - Channels may be different imaging modalities, like phase or fluorescence, or different markers, like nuclear or cytoplasm |
90 | | -- **feature** tells us which labeled features we are viewing |
91 | | - - Features may be [FILL IN] |
92 | | - |
93 | | -The next three rows tell us where the canvas is within the current frame. |
94 | | - |
95 | | -- **zoom** tells us how much the frame is zoomed in |
96 | | -- **viewing (x)** the visible horizontal range |
97 | | -- **viewing (y)** the visible vertical range |
98 | | - |
99 | | -These two rows tell us about visual settings |
100 | | - |
101 | | -- **highlight** tells us whether the label we're painting with is highlighted in red |
102 | | -- **brush size** tells us the radius of the brush in pixels |
103 | | - |
104 | | -The next two rows tell us about the label the mouse is hovering over |
105 | | - |
106 | | -- **label** is the label number under the mouse |
107 | | -- **slices** is a list of the frames that the label is present in |
108 | | - |
109 | | -The next two rows tell us which labels we can edit with the brush tool |
110 | | - |
111 | | -- **painting with label** is the label the we brush adds to the canvas |
112 | | -- **painting over label** is the label the brush overwrites |
113 | | - |
114 | | -The last two rows are about label editing actions |
115 | | - |
116 | | -- **tool** tells us what action happens when we click on the canvas |
117 | | - - See the [Tools](#Tools) section for more details |
118 | | -- **confirm action** is a prompt displayed before some actions |
119 | | - - We can confirm the action with Enter, or cancel it with Esc. See the [Actions](#Actions) section for more details. |
120 | | - |
121 | | -## Controls |
122 | | - |
123 | | -_These controls are also found in the dropdown "instructions" pane while editing a file._ |
124 | | - |
125 | | -### Navigating the Canvas |
126 | | - |
127 | | -Hover the mouse over a label to view the label info. |
128 | | - |
129 | | -Press <kbd>+</kbd> or hold <kbd>Alt and scroll up</kbd> to zoom in. |
130 | | - |
131 | | -Press <kbd>+</kbd> or hold <kbd>Alt and scroll down</kbd> to zoom out. |
132 | | - |
133 | | -Hold <kbd>Space</kbd> and move the mouse on the canvas to pan around. |
134 | | - |
135 | | -Press <kbd>D</kbd> or <kbd>→</kbd> to view the next frame. |
136 | | - |
137 | | -Press <kbd>A</kbd> or <kbd>←</kbd> to view the previous frame. |
138 | | - |
139 | | -Press <kbd>C</kbd> to view the next channel. |
140 | | - |
141 | | -Press <kbd>Shift</kbd> + <kbd>C</kbd> to view the previous channel. |
142 | | - |
143 | | -Press <kbd>F</kbd> to view the next feature. |
144 | | - |
145 | | -Press <kbd>Shift</kbd> + <kbd>F</kbd> to view the previous feature. |
146 | | - |
147 | | -### Adjusting the Canvas |
148 | | - |
149 | | -DeepCell Label can adjust how to display labels and images. |
150 | | -With these changes, we can make out finer details while labeling. |
151 | | - |
152 | | -Scroll on the canvas to change brightness. |
153 | | - |
154 | | -Hold <kbd>Shift</kbd> and scroll on the canvas to change the contrast. |
155 | | - |
156 | | -Press <kbd>0</kbd> (zero) to reset the brightness and contrast. |
157 | | - |
158 | | -Press <kbd>Z</kbd> to toggle between viewing the label overlay, the labels only, and the raw image only. |
159 | | - |
160 | | -Press <kbd>H</kbd> to toggle highlighting the brush label in red. |
161 | | - |
162 | | -## Select Labels |
163 | | - |
164 | | -We control which labels we cab edit by selecting them as |
165 | | -the <strong>foreground</strong> or the <strong>background</strong>. |
166 | | - |
167 | | -When a label is the foreground, we can add more of that label, |
168 | | -while when a label is background, we can remove it. |
169 | | - |
170 | | -You select "no label" as either the foreground, |
171 | | -which lets us remove existing labels, or the background, |
172 | | -which lets us labels over unlabeled areas. |
173 | | - |
174 | | -We can select a label as the foreground in two ways: |
175 | | - |
176 | | -<strong>Click</strong> on it while using the Select Tool |
177 | | - |
178 | | -<strong><kbd>Shift</kbd> + Double Click</strong> while using any Tool. |
179 | | - |
180 | | -We can select a label as the background in two ways: |
181 | | - |
182 | | -<strong>Double Click</strong> on it while using the Select Tool. |
183 | | - |
184 | | -<strong><kbd>Shift</kbd> + Click</strong> while using any Tool. |
185 | | - |
186 | | -Double clicking also deselects other labels, instead making "no label" the other selected label. |
187 | | -This helps avoid having selected labels that you are no longer working on. |
188 | | - |
189 | | -When you select an label that is already selected, |
190 | | -the foreground and background swap instead. |
191 | | - |
192 | | -Use this as a shortcut when working on a pair of labels, like adjusting their boundary |
193 | | -so you can quickly work in both directions. |
194 | | - |
195 | | -Here are some keyboard shortcuts to change the selected labels. |
196 | | - |
197 | | -- Press <kbd>Esc</kbd> to reset the background to "no label". |
198 | | -- Press <kbd>N</kbd> to select a new, unused foreground label. |
199 | | - - This will be a label that doesn't exist yet in the labeling. Once you add the new label, <kbd>N</kbd> will select a new, even higher label. |
200 | | -- Press <kbd>X</kbd> to swap the foreground and background. Remember that you can also swap them by selecting either label a second time with the mouse. |
201 | | -- Press <kbd>[</kbd> or <kbd>]</kbd> to cycle the foreground labels. |
202 | | -- Press <kbd>Shift</kbd> + <kbd>[</kbd> or <kbd>Shift</kbd> + <kbd>]</kbd> to cycle the background labels. |
203 | | - |
204 | | -## Tools |
205 | | - |
206 | | -Each tool lets us edit the labeling in a different way. To edit the labels with a tool, click on canvas while using the tool. |
207 | | - |
208 | | -### Select Tool |
209 | | - |
210 | | -Label starts with the Select tool by default, which selects the foreground and the background label. |
211 | | - |
212 | | -Press <kbd>V</kbd> to use the Select tool. |
213 | | - |
214 | | -<strong>Click</strong> on a label to select it as the foreground, |
215 | | - |
216 | | -<strong>Double Click</strong> on a label to select it as the background. |
217 | | - |
218 | | -You can also select labels while using <ital>any tool</ital> with |
219 | | -<strong><kbd>Shift</kbd> + Click</strong> to select the background, or <strong><kbd>Shift</kbd> + Double Click</strong> to select the foreground. |
220 | | - |
221 | | -Double Click to select also deselects other labels. |
222 | | - |
223 | | -### Brush Tool |
224 | | - |
225 | | -The Brush tool paints the foreground label over the background label. Other labels are left unedited. |
226 | | - |
227 | | -Press <kbd>B</kbd> to use the Brush tool. |
228 | | - |
229 | | -<strong>Click + Drag</strong> to paint. |
230 | | - |
231 | | -Press <kbd>↑</kbd> to increase the brush size. |
232 | | -Press <kbd>↓</kbd> to decrease the brush size. |
233 | | - |
234 | | -### Threshold Tool |
235 | | - |
236 | | -The Threshold tool fills the brightest pixels within a bounding box with the foreground label. The tool only overwrites unlabeled area and does not edit any other labels. |
237 | | - |
238 | | -Press <kbd>T</kbd> to use the Threshold tool. |
239 | | - |
240 | | -<strong>Click + Drag</strong> on the canvas to draw a bounding box and threshold. |
241 | | - |
242 | | -### Grow/Shrink Tool |
243 | | - |
244 | | -The Grow/Shrink tool expands the foreground label by one pixel or contracts the background label by one pixel. |
245 | | - |
246 | | -Press <kbd>Q</kbd> to use the Grow/Shrink tool. |
247 | | - |
248 | | -<strong>Click</strong> on the foreground label to grow it. |
249 | | - |
250 | | -<strong>Click</strong> on the background label to shrink it. |
251 | | - |
252 | | -### Flood Tool |
253 | | - |
254 | | -The Flood tool fills connected areas of label with another label. |
255 | | - |
256 | | -Press <kbd>G</kbd> to use the Flood tool. |
257 | | - |
258 | | -<strong>Click</strong> on the background label to flood it with the foreground label. |
259 | | - |
260 | | -### Trim Tool |
261 | | - |
262 | | -The Trim tool removes disconnected areas of the background label, leaving only the connected pixels behind. |
263 | | - |
264 | | -Press <kbd>K</kbd> to use the Trim tool. |
265 | | - |
266 | | -<strong>Click</strong> on the background label to trim it. |
267 | | - |
268 | | -### Autofit Tool |
269 | | - |
270 | | -The Autofit tool adjusts foreground label boundary to hug the nearest edges in the raw image. This tool helps to fix the boundaries of an existing label, but it can't add a new label from scratch. |
271 | | - |
272 | | -Press <kbd>M</kbd> to use the Autofit tool. |
273 | | - |
274 | | -<strong>Click</strong> on the foreground label to autofit it. |
275 | | - |
276 | | -### Watershed Tool |
277 | | - |
278 | | -The Watershed tool splits the foreground label into two. We can separate adjacent cells that mistakenly have the same label with Watershed. |
279 | | - |
280 | | -Press the <kbd>W</kbd> to use the Watershed tool. |
281 | | - |
282 | | -<strong>Click</strong> in the center of one cell, then <strong>Click</strong> in the center of another cell elsewhere in the same label to split them with Watershed. |
283 | | - |
284 | | -### Tools with Select Shortcuts |
285 | | - |
286 | | -Tools like Autofit or Flood can only edit selected labels. |
287 | | - |
288 | | -To make these tools easier to use, clicking on an unselected label with these tools selects the label instead of doing an action. After clicking once to select, a second click on the same label always uses the tool and edits the labels. |
289 | | - |
290 | | -The Grow/Shrink, Autofit, and Watershed tools makes unselect labels the foreground. |
| 43 | +When starting the application, Flask will create the database and tables if they do not exist. By default, the app makes a database in `/tmp` and erases on restart. To make a persistent database, change `SQLALCHEMY_DATABASE_URI` in `.env` to create the database in another folder like `SQLALCHEMY_DATABASE_URI=sqlite:///~/Documents/deepcell_label.db`. |
291 | 44 |
|
292 | | -The Trim and Flood tools makes unselected labels the background. |
| 45 | +### Set up React client |
293 | 46 |
|
294 | | -## Actions |
| 47 | +Once the server is running, we need to set up the frontend to interact with it. Install the dependencies for the frontend with: |
295 | 48 |
|
296 | | -Actions edit labels with a keybind. |
297 | | -Actions need to be confirmed with Enter or cancelled with Escape. |
| 49 | +``` |
| 50 | +cd visualizer |
| 51 | +yarn |
| 52 | +``` |
298 | 53 |
|
299 | | -Press <kbd>S</kbd> to swap the label foreground and background labels. |
300 | | -This action can only swap the labels on the current frame. |
| 54 | +Then, start the frontend with: |
301 | 55 |
|
302 | | -Press <kbd>R</kbd> to replace the background label with the foreground label. |
303 | | -Press <kbd>Shift</kbd> + <kbd>R</kbd> to replace the background label on all frames. |
| 56 | +``` |
| 57 | +yarn start |
| 58 | +``` |
304 | 59 |
|
305 | | -Press <kbd>O</kbd> to predict the labels based on the overlap with the previous frame. |
306 | | -Press <kbd>Shift</kbd> + <kbd>O</kbd> to predict the labels on all frames. |
| 60 | +Visit [localhost:3000](http://localhost:3000) to see the DeepCell Label homepage. |
0 commit comments