Ruby Google Scraper API endpoints

Available APIs on Ruby Google Scraper app

We're following JSON API specification.

Sign up

POST /api/v1/users?username=nimblehq&password=123456&password_confirmation=123456

Possible response example

201 Created

{
    "data": {
        "id": "1",
        "type": "user",
        "attributes": {
            "username": "nimblehq"
        },
        "relationships": {
            "keywords": {
                "data": []
            }
        }
    }
}

400 Bad Request

{
    "errors": [
        {
            "detail": "Username has already been taken"
        }
    ]
}

---

Sign in (Request for access token)

# User credentials
POST /api/v1/oauth/token?client_id={{client_id}}&client_secret={{client_secret}}&grant_type=password&username=nimblehq&password=123456

# Refresh token
POST /api/v1/oauth/token?client_id={{client_id}}&client_secret={{client_secret}}&grant_type=refresh_token&refresh_token={{refresh_token}}

Possible response example

200 OK

{
    "data": {
        "id": 1,
        "type": "token",
        "attributes": {
            "access_token": "hybTAeocW3YusPBYtn-LQ5w1w2D1fx3MdBJ6Wmb5HyU",
            "token_type": "Bearer",
            "expires_in": 7200,
            "refresh_token": "WUQONLDiCy0iiLW3nxR0EZdsxmTHRJw3CUWjCLjB1VU",
            "created_at": 1604049023
        }
    }
}

400 Bad Request

{
    "errors": [
        {
            "source": "Doorkeeper::OAuth::Error",
            "detail": "The provided authorization grant is invalid, expired, revoked, does not match the redirection URI used in the authorization request, or was issued to another client.",
            "code": "invalid_grant"
        }
    ]
}

401 Unauthorized

{
    "errors": [
        {
            "source": "Doorkeeper::OAuth::Error",
            "detail": "Client authentication failed due to unknown client, no client authentication included, or unsupported authentication method.",
            "code": "invalid_client"
        }
    ]
}

---

Sign out (Revoke access token)

POST /api/v1/oauth/revoke?client_id={{client_id}}&client_secret={{client_secret}}&token={{access_token}}

Possible response example

200 OK

{}

403 Forbidden

{
    "errors": [
        {
            "detail": "You are not authorized to revoke this token",
            "code": "unauthorized_client"
        }
    ]
}

---

Fetch keywords (limit 25 keywords)

# Default page is 1
GET /api/v1/keywords

# Specific page
GET /api/v1/keywords?page=2

# Search keyword
GET /api/v1/keywords?search=Cristiano

# Header
Authorization: Bearer {{access_token}}

Possible response example

200 OK
# Keywords are ascending ordered by keyword

{
  "data": [
    {
        "id": "3",
        "type": "keyword",
        "attributes": {
            "keyword": "Cristiano Ronaldo",
            "status": "completed",
            "top_pos_adwords": 0,
            "adwords": 0,
            "non_adwords": 2,
            "links": 126,
            "html_code": "<!DOCTYPE...",
            "top_pos_adword_links": [],
            "non_adword_links": [
                "https://en.wikipedia.org/wiki/Cristiano_Ronaldo",
                "https://www.instagram.com/cristiano/?hl=th"
            ],
            "failed_reason": null
        },
        "relationships": {
            "user": {
                "data": {
                    "id": "1",
                    "type": "user"
                }
            }
        }
    },
    {
        "id": "1",
        "type": "keyword",
        "attributes": {
            "keyword": "Eden Hazard",
            "status": "completed",
            "top_pos_adwords": 0,
            "adwords": 0,
            "non_adwords": 9,
            "links": 122,
            "html_code": "<!DOCTYPE...",
            "top_pos_adword_links": [],
            "non_adword_links": [
                "https://en.wikipedia.org/wiki/Eden_Hazard",
                "https://www.instagram.com/hazardeden_10/?hl=th"
            ],
            "failed_reason": null
        },
        "relationships": {
            "user": {
                "data": {
                    "id": "1",
                    "type": "user"
                }
            }
        }
    }
  ]
}

200 OK

{
    "data": []
}

401 Unauthorized

{
    "errors": [
        {
            "source": "unauthorized",
            "detail": "The access token is invalid",
            "code": "invalid_token"
        }
    ]
}

Response Explanation

• Status
  • in_queue: Keyword is in the job queue
  • completed: Scrap from google successfully
  • failed: Scrap from google unsuccessfully

---

Fetch keyword scraping result

GET /api/v1/keywords/{{keyword_id}}

# Header
Authorization: Bearer {{access_token}}

Possible response example

200 OK

{
  "data": {
    "id": "1",
    "type": "keyword",
    "attributes": {
        "keyword": "Eden Hazard",
        "status": "completed",
        "top_pos_adwords": 0,
        "adwords": 0,
        "non_adwords": 9,
        "links": 122,
        "html_code": "<!DOCTYPE...",
        "top_pos_adword_links": [],
        "non_adword_links": [
            "https://th.wikipedia.org/wiki/%E0%B9%80%E0%B8%AD%E0%B9%81%E0%B8%94%E0%B8%99_%E0%B8%AD%E0%B8%B2%E0%B8%8B%E0%B8%B2%E0%B8%A3%E0%B9%8C",
            "https://en.wikipedia.org/wiki/Eden_Hazard",
            "https://www.transfermarkt.com/eden-hazard/profil/spieler/50202",
            "https://www.instagram.com/hazardeden_10/?hl=th",
            "https://www.pinterest.com/poon_tbsbabor/eden-hazard/",
            "https://www.pinterest.com/mayurespu/eden-hazard/",
            "https://sites.google.com/site/chelseaionbulecom/ede",
            "https://twitter.com/hazardeden10",
            "https://th-th.facebook.com/edenhazard/"
        ],
        "failed_reason": null
    },
    "relationships": {
        "user": {
            "data": {
                "id": "1",
                "type": "user"
            }
        }
    }
  }
}

401 Unauthorized

{
    "errors": [
        {
            "source": "unauthorized",
            "detail": "The access token is invalid",
            "code": "invalid_token"
        }
    ]
}

404 Not Found

{
    "errors": [
        {
            "detail": "Couldn't find Keyword with 'id'=32"
        }
    ]
}

---

Upload CSV File

POST /api/v1/keywords

# Header
Authorization: Bearer {{access_token}}

# Body (form)
file={{path_to_your_file}}

Possible response example

204 No Content

400 Bad Request

{
    "errors": [
        {
            "detail": "File must be CSV format"
        }
    ]
}

401 Unauthorized

{
    "errors": [
        {
            "source": "unauthorized",
            "detail": "The access token is invalid",
            "code": "invalid_token"
        }
    ]
}