Where's The Data?
data management
data management
I received an interesting email from someone in France today about the Blood Sugars project. I live in upstate New York. The Internet is AWESOME. I have not posted any of this to any of the R meme-blogo-spheres, so I wonder how she found my little project. It doesn’t really matter, but I do intend to ask. My new friend is a Type 1 diabetic who would like to follow along with the code examples I have been posting but isn’t sure how to start.
Today’s Objectives: Getting Started
In today’s post, I will try to answer some questions to new users get started with the code I’m writing here. Example questions include:
- How can I get the data?
- How to follow along at home?
- Why is the sky blue?
The first two are great questions that we will discuss today. I’m sure Wikipedia can help you with the third question. The ugly truth is, learning how to run analytical software is hard. You won’t figure it all out today, or this week. Take it one step at a time, read everything you can get your hands on and have fun.
To my new French pen pal, I am sending you everything you need in an email.
There are two really good R-documentation websites on the Internet that I like to point people to. I use these all the time.
If you are looking for programming help, Stack Overflow is a well organized, polite place to get AMAZING help. The people there are scary-smart. R-specific help can be found at:
https://stackoverflow.com/questions/tagged/r
If you are new to R, bookmark all three. All of the code and data I am creating is hosted on my GitHub account. The data should be usable in ANY analysis tool capable of importing CSV files. The repository page can be found here:
https://github.com/Choens/blood-sugars.
The first post on this project blog was on July 29, 2015. In the post, “Get CGM Data”, I present some code to import Nightscout data into R and save it as a CSV file. The R Markdown file (and thus the code) used to create the blog post can be found here:
https://github.com/Choens/blood-sugars/blob/master/2015-07-29-get-cgm-data.Rmd.
The data is stored in the data folder of the project. If you would like to see all of our public data files, they are available here:
https://github.com/Choens/blood-sugars/tree/master/data
As of September 1, 2015, there are two CSV data sets in the data folder. To make life easier for everyone, the following two links will take you to the raw CSV files. If you want to have some CGM data, right click on either of the following two links and save.
The file, “entries-2015-07-29.csv” was produced as a side-affect of compiling the R Markdown code which produced the blog post for July 29. The second file is a larger data set which I produced today using a small function and script I added to make it easier to produce new data sets.
The first file contains all of our CGM data as of July 29, 2015. The second file contains all of our CGM data as of September 1, 2015. Each file-name contains a date stamp which corresponds with when the files were created. Each file is a complete extract of our Nightscout database as of the date shown. Obviously, these files overlap. You do not have to concatenate them to get a complete data set.
The first row contains the column names. To use the data files, you do not have to concatenate the two files. Import either CSV file into your favorite analysis tool and have some fun. The data is ONLY CGM data. It does not include any exercise or calorie information. That is good information to collect, but isn’t part of the Nightscout project.
If you want to grab all of the code, data, etc. and don’t want to wade though all the links I just gave you, all you need to do is right click on the link below and save the master.zip file to your local computer. Caveat emptor, but none of the code here will destroy your computer. I promise.
https://github.com/Choens/blood-sugars/archive/master.zip
The master.zip file contains the current “master” branch of the project. Unzip the folder anywhere you want. The CGM data is in the “data” folder. R functions are in the “R” folder. R Markdown files and scripts are in the root folder.
If you are a Git user, you can download the repository using Git by cloning: https://github.com/Choens/blood-sugars.git
If you want to use the R code, make sure you set your working directory to the folder where the R Markdown files (.Rmd) are. This is the working directory I use when writing the code. If you aren’t sure what that means, read this.
If you are new to R, I STRONGLY recommend you use RStudio. It is the best R programming environment for new programmers. It is much better than the native R interface. RStudio is easy to set up, easy to use and is freely available for Linux, Mac and Windows. And by free, I mean as in speech. This is open source and it is GOOD software.
Setting the R working directory is really easy in RStudio. Open any of the .Rmd files in the folder where you unzipped the master.zip file. In the menu, select:
“Session” –> “Set Working Directory” –> “To Source File Location”
Congrats. You are done. Your working directory is now set to the folder holding the Blood Sugar .Rmd files. All of the code should now work.
If the code doesn’t work and you think my code is broken; file a bug against the project here. If my writing is unclear, feel free to send me an email.