The processing of flavor
Synchronizing DynamoDB and MySQL

Synchronizing DynamoDB and MySQL

When data comes into our system, it is put into Amazon’s DynamoDB, which is efficient for most of our operations. The Gastrograph app shows users their previous reviews, and this data is easy to access using the username as a hash key and the review time as a range key. Nevertheless, our system depends on data processing, which is done in R and is much easier using SQL. Therefore, we need a copy of our raw data in MySQL which is then filtered and processed by our machine learning algorithms.

Read more
Website downtime on May 22, 2015

If you tried to reach this blog or website between 1:21 PM EST and ~6:24 PM EST on Friday, May 22, 2015, your browser may have warned you that uses an "invalid security certificate".

Read more
Update: Twilio And Shiny Integrated

Since we use Twilio in a lot of our R packages while running analysis for our clients, we have built a wrapper for contacting the API. You can see and install it here. We have also built API wrappers for Stripe, Sendwithus, and Mandrill. Using these helps prevent code bloat and makes it cleaner. A lot of this post is going to be very similiar to our old blog post about Twilio and Shiny.

As Shiny Server gets more and more production ready, developers are adding new features that typically aren't easy to build in—examples include automated phone notifications through Twilio. Twilio allows your application to send phone calls and text messages to users of your application when a triggering event occurs. For example, we use it to call our clients of potential flaws, taints, and contamination's in their products in real time.

This tutorial will teach you to integrate Twilio with a basic Shiny application which places a sample call to your users. This tutorial will leave you with a working application which you can build into your own needs. This tutorial will require the following:

  • R
  • Shiny

This tutorial focuses on Shiny Server, but most of the code could easily be converted to other R based web frameworks or batch scripts. The code for the Shiny Version is located on GitHub.

Read more
Using S3 buckets in R with RS3 library

In the past there hasn't been a good location for storing flat files as an R developer. If you only run a program on your local machine, storing it there isn't a big problem. Once you start needing backups and access from multiple locations, sending the updated file via email isn't easy and worth it. You can run your own FTP server to solve this problem, but then you run higher running costs, the need for maintenance, backups, and security.

The other solution is to use AWS S3 buckets. These buckets are easy to use, simple to get started with, cheap at $0.03 per GB of storage, redundant (so you won't lose the files), and are easy to create access restrictions for. While running applications, you could store any CSV files you need on S3, or any other information without needing to access a database repeatedly. There are many advantages to using S3 buckets. In the past, the biggest problem for using S3 buckets with R was the lack of easy to use tools.

Read more
Setting up LDAP for use in Shiny

LDAP, or Lightweight Directory Access Protocol, has a long history of use. LDAP is usually used in large enterprise corporations where they store employee data and use it for single sign-on (SSO) and more. Single sign-on is where the company's LDAP directory integrates with other applications such as the company intranet using the protocol.

For companies using R as the programming language of choice, Shiny Server is incredibly useful as a server. Unfortunately, when authentication needs put in place, there are only a few options:

  1. Flat file—good if you don't add new users often and you don't want to have to reset the password.
  2. PAM—you need to know Linux well to set up, and it is restricting for a large user base like flat-file.
  3. Google—good if you know all your clients have an account.
  4. LDAP—until Shiny accepts using a database connection, LDAP seems like the best choice.

Getting a LDAP server running can be very tedious; in order to set up it up, you need to learn a lot about how the convoluted ldif files work with adding new entries. And, most of the information on the web can be limited on what server package to actually use. Here is how we set up our LDAP server to authenticate our clients.

Read more

Looking for a specific post?