Organizing Your Data

Creating a folder structure | File naming | Version control | Resources

Creating a system for organizing and managing your files at the beginning of a project will make it easier to search for, find, and work with your research data. There is no one right way to organize and name your files, as this will depend on what makes the most sense for your project. However, there are some best practices that can help you develop a system that works for you.

Creating a folder structure

A good folder structure will allow you to keep your files organized and easy to find.

Think about the organizing principles that make sense for your data. Consider how you or your research group will find, access and use your files. For example, your data can be organized by project, content, location, date, or something else.
Create an easy-to-navigate structure. The number of folders and subfolders you need will depend on your project - large projects may need many while smaller projects may only need a few. Sub-folders can facilitate navigation, but if they're not well organized they can become confusing and hard to navigate.
Take advantage of shortcuts. Use shortcuts instead of duplicating folders and files in multiple places. This can make it easier to quickly move between related folders while preventing messy and inconsistent data files.
Document your folder structure. Document any information someone would need to navigate your folder structure in a README file. A README is a plain text file that provides information about the content and organization of the folders and files it accompanies. A README should include the folder structure, file naming conventions, and any other relevant information needed to work with the files.

File naming

How you name your files impacts how you are able to find and work with them. Develop and document a file naming system at the beginning of your project and use it consistently.

Note: if you're working with sensitive data, do not include sensitive information in your file names (e.g., a person's name).

File sorting

It may be useful to sort your files by date, numerical sequence, alphabetically, or by some other system. Think about what information to include in your file names (including in what order and format) so you can sort them in a way that makes sense for your project.

Use date format YYYYMMDD (ISO 8601). If you want to sort your files by date, use the format YYYYMMDD so they sort in proper chronological order. Establishing a standard date format can also help avoid confusion and ensure everyone in your research group is using the same system. For example, without establishing a standard format 20230304 could be interpreted as March 4, 2023 or April 4, 2023.
Use leading zeros. If you decide to use numbers in your file names you can use leading zeros to ensure they sort appropriately. For a sequence of 1-100 use 001-100.

Opening and executing files

How you name your files may impact how you are able to open and execute them.

Avoid special characters like & , * % . # * ( ) ! @$ ^ ~ ‘ { } [ ] ? < > Using special characters may cause problems when opening files in certain software or when using the command line.
Separate words or concepts using underscores or CamelCase. Using periods or spaces in file names can cause problems if you are using the command line. An alternative is to use underscores or CamelCase (capitalizing the first letter of each new word or concept, e.g., TransportationSurveyData2021).
Use 32 or fewer characters. Long file names can be difficult to preview or read. Additionally, some programs may not be able to open files with names longer than 32 characters.

Identifying and keeping track of your files

Create a file naming system that allows you to identify your files without needing to open them.

Use meaningful and unique file names. A meaningful name means you should be able to tell what is in the file without opening it. A unique name means no two files have the same name, even if they are in different folders. Using meaningful and unique file names will allow you to identify your files even if you remove them from the folder structure (e.g., if you are working with your files in a software environment).
Avoid using terms like “final'' or “revised” to distinguish between different versions for a file. Instead, include a version control element like v1, v2, etc. You can also use version control elements to distinguish between major file updates (v1, v2) and minor file updates (v1_1, v1_2).

Version control

Version control is the practice of managing and retaining different iterations or revisions of files over time. It helps ensure you are working with the correct version of a file and supports collaborative work. Creating a new version of a data file after a major update can help distinguish backup points and identify errors between versions. It can also help you keep a record of changes to your files over time, go back and check your work, and retrieve data in case of data loss.

Version control may be managed manually or through version control software.