Build Up a Homemade Server-to-server Interactions With Google API Using Ruby - Take Google Analytics as an Example

Since ‘google-api-ruby-client’ make a big change from 0.8 to 0.9 when my project is running out of time, and also I met this issue, So, I make this decision, let’s build a API client by myself.

This implementation refer to Google guideline

Step1. Preparation

1. Create a Service Account

add credential

2. Download JSON file

Download JSON File

And I put JSON file into config folder.

3. Copy a Email and Add to Google Analytics

Add generated email to Google analytics user

Step2. Get Access Token

Add to Gemfile

1
2

gem "typhoeus"
gem 'jwt'

Create a file named lib/google_anallytics.rb

Again, this implementation refer to Google guideline. Please, check this guideline first. Then you will understand how I write fuction get_token.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40

require 'typhoeus'
require "jwt"
require 'date'
module GoogleAnalytic
 
  def get_token
    json_file = File.join("#{Rails.root}/config", 'YOUR-JSON-FILE-NAME.json')
    file = File.read(json_file)
    key_hash = JSON.parse(file)
    aud     = "https://www.googleapis.com/oauth2/v3/token"
    now     = Time.new
    iat     = (now - 60).to_i
    @exp    = (now + 50.minutes).to_i
    scopes  = 'https://www.googleapis.com/auth/analytics.readonly'
    sub     = key_hash["client_email"]
    iss     = key_hash["client_email"]
    jwt_claim_set    = {
      'iss' => iss, 
      'sub' => sub, 
      'scope' => scopes,
      'aud' => aud, 
      'exp' => @exp,
      'iat' => iat
      }
    token            = JWT.encode(jwt_claim_set,  OpenSSL::PKey::RSA.new(key_hash["private_key"]), 'RS256')
    
    res = Typhoeus::Request.post('https://www.googleapis.com/oauth2/v3/token', 
      body: 
        { 
          grant_type: "urn:ietf:params:oauth:grant-type:jwt-bearer",
          assertion: token
        }
    )
    
    res_hash = MultiJson.load(res.body)

    token = res_hash["access_token"]
    return token
  end
end

Step3. Fetch Data

In my case, I create a model for saving Google Analytics data. So, I’ll show you how I fetch data.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71

class GoogleAnalyticEvent
  include GoogleAnalytic
  include Mongoid::Document
  include Mongoid::Timestamps
  field :category, type: String        
  field :label, type: String        
  field :action, type: String
  field :total_events, type: Integer       
  field :unique_events, type: Integer
  field :data_date, type: Date
  index({category: 1, label: 1, data_date: 1, action:1 })
  
  def get_event_from_ga_api(params={})
    @data_date = Date.today - 1
    @parameters = {
        'ids'         => "ga:12345678",
        'start-date'  => @data_date.strftime("%Y-%m-%d"),
        'end-date'    => @data_date.strftime("%Y-%m-%d"),
        'metrics'     => "ga:totalEvents,ga:uniqueEvents",
        'dimensions'  => "ga:eventCategory,ga:eventLabel,ga:eventAction",
        'max-results' => "10000"
      }
    query_paginate(@parameters)
  end
  
  def query_paginate(parameters)
    parameters['access_token'] = get_token
    count = 0
    loop do
        if Time.new.to_i > @exp
          parameters['access_token'] = get_token
        end
       res = Typhoeus::Request.get('https://www.googleapis.com/analytics/v3/data/ga', 
        params: parameters
       )
      result = MultiJson.load(res.body)
      if result["error"].nil?
        sync_to_db(result)  if result["totalResults"] > 0
        max_count = result["totalResults"] / parameters["max-results"].to_i
      end
      break if count == max_count || result["error"].present?
      count += 1
      parameters["start-index"] = parameters["max-results"].to_i * count + 1
    end
  end

  def sync_to_db(result)
    result["columnHeaders"].each_with_index do |h,i|
      case h["name"].to_s
      when "ga:eventCategory"
        @category= i 
      when "ga:eventLabel"
        @label   = i
      when "ga:eventAction"
        @action  = i
      when "ga:totalEvents" 
        @total_events  = i
      when "ga:uniqueEvents"
        @unique_events = i
      end
    end 
    result["rows"].each do |array_data|
        GoogleAnalyticEvent.create(category:      array_data[@category], 
                                    label:        array_data[@label], 
                                    action:       array_data[@action],
                                    total_events:       array_data[@total_events],
                                    unique_events:      array_data[@unique_events],
                                    data_date:          @data_date)
    end
  end
end

Let’s me explain each code snapshots.

3-1: First of all, I initialize query parameters such as ids, start-date, …etc in instance variable @parameters.

PS. you can easily understand which parameters you need by using Google Analytics Query Explorer. Once, you find out which parameters you need, just copy to here like I do.

1
2
3
4
5
6
7
8
9
10
11
12

def get_event_from_ga_api(params={})
    @data_date = Date.today - 1
    @parameters = {
        'ids'         => "ga:12345678",
        'start-date'  => @data_date.strftime("%Y-%m-%d"),
        'end-date'    => @data_date.strftime("%Y-%m-%d"),
        'metrics'     => "ga:totalEvents,ga:uniqueEvents",
        'dimensions'  => "ga:eventCategory,ga:eventLabel,ga:eventAction",
        'max-results' => "10000"
      }
    query_paginate(@parameters)
  end

3-2: Second, let’s see sync_to_db first.

After I got Google API’s response, I will save to database.

let’s see what do we get from google analytics first.

Google Analytics Return Data

Google show you the return data’s column header name by “columnHeaders”. And “rows” data is given as an array. And each array’s elements refer to “columnHeaders” respectively. For exameple, in above picture, you can see “ga:eventCategory” is in first of columnHeaders. So, in rows, each array’s first element is value of “ga:eventCategory”. Then I save data to model GoogleAnalyticEvent.

That’s all fuction “sync_to_db” do.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24

def sync_to_db(result)
    result["columnHeaders"].each_with_index do |h,i|
      case h["name"].to_s
      when "ga:eventCategory"
        @category= i 
      when "ga:eventLabel"
        @label   = i
      when "ga:eventAction"
        @action  = i
      when "ga:totalEvents" 
        @total_events  = i
      when "ga:uniqueEvents"
        @unique_events = i
      end
    end 
    result["rows"].each do |array_data|
        GoogleAnalyticEvent.create(category:      array_data[@category], 
                                    label:        array_data[@label], 
                                    action:       array_data[@action],
                                    total_events:       array_data[@total_events],
                                    unique_events:      array_data[@unique_events],
                                    data_date:          @data_date)
    end
  end

3-3: Finally, let’s see function query_paginate.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20

def query_paginate(parameters)
    parameters['access_token'] = get_token
    count = 0
    loop do
        if Time.new.to_i > @exp
          parameters['access_token'] = get_token
        end
       res = Typhoeus::Request.get('https://www.googleapis.com/analytics/v3/data/ga', 
        params: parameters
       )
      result = MultiJson.load(res.body)
      if result["error"].nil?
        sync_to_db(result)  if result["totalResults"] > 0
        max_count = result["totalResults"] / parameters["max-results"].to_i
      end
      break if count == max_count || result["error"].present?
      count += 1
      parameters["start-index"] = parameters["max-results"].to_i * count + 1
    end
  end

I have to get access token by “get_token”, And this function also assign expire time to @exp.

1

parameters['access_token'] = get_token

Then I write a loop for pagination. At first, I check token is expired or not, if is expired, I have to get token again.

1
2
3

if Time.new.to_i > @exp
  parameters['access_token'] = get_token
end

Now, I use Typhoeus::Request.get method with params fetching data from Google Analytics.

1
2
3
4

res = Typhoeus::Request.get('https://www.googleapis.com/analytics/v3/data/ga', 
        params: parameters
       )
result = MultiJson.load(res.body)

And if there are no any errors, I will save data to database.

1
2
3
4

if result["error"].nil?
  sync_to_db(result)  if result["totalResults"] > 0
  max_count = result["totalResults"] / parameters["max-results"].to_i
end

Finally, break or continue to get next batch data.

1
2
3

break if count == max_count || result["error"].present?
count += 1
parameters["start-index"] = parameters["max-results"].to_i * count + 1

That’s it.

Google Analytics, Ruby on Rails
Comments

Comments

Google Analytics Alternative