FinallyBot – A Twitterbot that finds all the tech news that is finally here

FinallyBotMy first foray into creating a twitterbot is FinallyBot.  A bot that scours restless tech sites for news that has finally happened.

It tracks RSS feeds for about 120 technology sites.  If it finds a news story with “finally” in the title, it generates a short link with bit.ly (using BitlyPHP), and tweets it (using Twitter for PHP)

It’s been quietly running for about 8½ months now and has over 1700 tweets.  That’s about 200 news items with “finally”  in the title per month.

The most impatient site? Averaging about one finally news item every other day, the honor goes to Gizmodo with 142 “finally” tweets in the past 36 weeks.

Site Tweets (past 36 weeks)
Gizmodo 142
Boy Genius Report 107
VentureBeat 77
Business Insider 68
Android Police 66
Engadget 57
The Next Web 51
The Verge 50
Quartz 50
Techcrunch 49

Whistle Dog Activity Monitor undocumented API

I, or I should say my dog, received a Whistle Activity Monitor as a gift. I’ve been a Fitbit user for a while, so this seemed like an interesting experiment.  (My dog is now wearing the Whistle and a Fitbit Zip on her collar so I can compare the two.)

Ketzel with her Whistle and Fitbit
Ketzel with her Whistle and Fitbit

So far, the Whistle is a pretty decent product, though expensive and limited.  At the moment, the only way to access information about your dog is through the iPhone/iOS app.  They don’t provide any interface to view the data on the web, no API, and no data export utility.

I spent some time this evening trying to figure out if I could pull the raw data directly from the Whistle server (to make it easier to compare to the Fitbit data, and maybe build a web page)

Using Charles Proxy I was able to view the HTTPS requests being made by the Whistle iOS app. You can mimic these calls (using something like Request Maker, cURL, or your own script) Here are raw details for logging in and requesting data:

Login

Request

https://app.whistle.com/api/tokens.json
POST /api/tokens.json HTTP/1.1
Host: app.whistle.com
Proxy-Connection: keep-alive
Accept: */*
Accept-Encoding: gzip, deflate
Content-Length: 179
Content-Type: application/json
Accept-Language: en;q=1, fr;q=0.9, de;q=0.8, ja;q=0.7, nl;q=0.6, it;q=0.5
Connection: keep-alive
User-Agent: WhistleApp/102 (iPhone; iOS 7.0.4; Scale/2.00)

{"password":"MY_WHISTLE_PASSWORD,"email":"MY_EMAIL_ADDRESS","app_id":"com.whistle.WhistleApp"}

Response

{"success":true,"token":"a_response_token_will_be_here"}

The token in the login response will then be passed in the “X-Whistle-AuthToken” HTTP header for followup requests.  Note, this is logging in with a Whistle account.  I think you can also login with a Facebook and some other 3rd party account.  I didn’t try this.

Get information about your dog

Request

https://app.whistle.com/api/dogs
 GET /api/dogs HTTP/1.1
 Host: app.whistle.com
 User-Agent: WhistleApp/102 (iPhone; iOS 7.0.4; Scale/2.00)
 Proxy-Connection: keep-alive
 Accept: */*
 Accept-Encoding: gzip, deflate
 Content-Type: application/json
 Accept-Language: en;q=1, fr;q=0.9, de;q=0.8, ja;q=0.7, nl;q=0.6, it;q=0.5
 Connection: keep-alive
 X-Whistle-AuthToken: your_token_that_was_returned_in_the_login_response

Response (JSON)

[{
 "id": "9999",
 "city_id": "12345",
 "date_of_birth": "2005-01-01",
 "gender": "f",
 "name": "Your Dog's Name",
 "weight": 75.0,
 "breed_name": "Dog Breed",
 "photo_url": "https://dog-photos-production.s3.amazonaws.com/dog-9999-photo/abcdef.jpg",
 "photo_url_sizes": {
 "30x30": "https://dog-photos-production.s3.amazonaws.com/dog-9999-photo/_30x30_abcdef.jpg",
 "60x60": "https://dog-photos-production.s3.amazonaws.com/dog-9999-photo/_60x60_abcdef.jpg",
 "640x1136": "https://dog-photos-production.s3.amazonaws.com/dog-9999-photo/_640x1136_abcdef.jpg",
 "640x960": "https://dog-photos-production.s3.amazonaws.com/dog-9999-photo/_640x960_abcdef.jpg",
 "320x480": "https://dog-photos-production.s3.amazonaws.com/dog-9999-photo/_320x480_abcdef.jpg"
 },
 "location": "City, ST",
 "profile_photo_url": "https://dog-photos-production.s3.amazonaws.com/dog-9999-photo/profile_ghijkl.jpg",
 "profile_photo_url_sizes": {
 "30x30": "https://dog-photos-production.s3.amazonaws.com/dog-9999-photo/_30x30_profile_ghijkl.jpg",
 "60x60": "https://dog-photos-production.s3.amazonaws.com/dog-9999-photo/_60x60_profile_ghijkl.jpg",
 "166x166": "https://dog-photos-production.s3.amazonaws.com/dog-9999-photo/_166x166_profile_ghijkl.jpg",
 "332x332": "https://dog-photos-production.s3.amazonaws.com/dog-9999-photo/_332x332_profile_ghijkl.jpg"
 },
 "relationship": "owner",
 "device_id": "3456",
 "current_activity_goal": {
 "minutes": 90,
 "started_at": "2014-01-01T00:00:00-05:00 America/New_York"
 }
}]

Get status data about the Whistle device

Request

https://app.whistle.com/api/devices/3456
GET /api/devices/3456 HTTP/1.1
Host: app.whistle.com
User-Agent: WhistleApp/102 (iPhone; iOS 7.0.4; Scale/2.00)
Proxy-Connection: keep-alive
Accept: */*
Accept-Encoding: gzip, deflate
Content-Type: application/json
Accept-Language: en;q=1, fr;q=0.9, de;q=0.8, ja;q=0.7, nl;q=0.6, it;q=0.5
Connection: keep-alive
X-Whistle-AuthToken: your_token_that_was_returned_in_the_login_response

Response (JSON)

{
 "serial_number": "W06-010ABC",
 "firmware_version": "1.0-131021 :: 1.1-131021 :: 1.10-131119-a",
 "battery_level": 42.750503100560245,
 "last_check_in": "2014-01-17T20:15:39-05:00 America/New_York",
 "next_check_in": "2014-01-17T21:15:39-05:00 America/New_York",
 "bluetooth_mac": null,
 "wifi_mac": null,
 "battery_days_left": 5.404172523606706
}

Get metric data

Request

https://app.whistle.com/api/dogs/9999/metrics?date=2014-01-03&id=9999
GET /api/dogs/9999/metrics?date=2014-01-03&id=9999 HTTP/1.1
Host: app.whistle.com
User-Agent: WhistleApp/102 (iPhone; iOS 7.0.4; Scale/2.00)
Proxy-Connection: keep-alive
Accept: */*
Accept-Encoding: gzip, deflate
Content-Type: application/json
Accept-Language: en;q=1, fr;q=0.9, de;q=0.8, ja;q=0.7, nl;q=0.6, it;q=0.5
Connection: keep-alive
X-Whistle-AuthToken: your_token_that_was_returned_in_the_login_response

Response (JSON)

{
 "date": "2014-01-03T00:00:00-05:00",
 "summary": {
 "total_minutes_active": 84,
 "total_minutes_rest": 1089
 },
 "events": [{
 "id": "10426",
 "start_time": "2014-01-03T07:04:30-05:00 America/New_York",
 "end_time": "2014-01-03T07:43:30-05:00 America/New_York",
 "event_type": "Walk",
 "event_title": "39 minutes",
 "event_details": null,
 "minutes_active": 39,
 "manually_created": false,
 "users": ["you"],
 "comments": [],
 "comments_count": 0
 }, {
 "id": "10610",
 "start_time": "2014-01-03T09:51:31-05:00 America/New_York",
 "end_time": "2014-01-03T10:19:31-05:00 America/New_York",
 "event_type": "Unknown",
 "event_title": "14 minutes",
 "event_details": null,
 "minutes_active": 14,
 "manually_created": false,
 "users": ["SpouseName"],
 "comments": [],
 "comments_count": 0
 }, {
 "id": "11448",
 "start_time": "2014-01-03T17:06:32-05:00 America/New_York",
 "end_time": "2014-01-03T17:26:34-05:00 America/New_York",
 "event_type": "Walk",
 "event_title": "14 minutes",
 "event_details": null,
 "minutes_active": 14,
 "manually_created": false,
 "users": [],
 "comments": [],
 "comments_count": 0
 }, {
 "id": "12195",
 "start_time": "2014-01-03T21:15:35-05:00 America/New_York",
 "end_time": "2014-01-03T21:33:35-05:00 America/New_York",
 "event_type": "Walk",
 "event_title": "17 minutes",
 "event_details": null,
 "minutes_active": 17,
 "manually_created": false,
 "users": ["you", "SpouseName"],
 "comments": [],
 "comments_count": 0
 }],
 "point_entries": [{
 "timestamp": "2014-01-03T00:00:00-05:00",
 "measurement": 0
 }, {
 "timestamp": "2014-01-03T00:30:00-05:00",
 "measurement": 0.9297725
 }, {
 "timestamp": "2014-01-03T01:00:00-05:00",
 "measurement": 0.0
 }, {
 "timestamp": "2014-01-03T01:30:00-05:00",
 "measurement": 26.538085939
 }, {
 "timestamp": "2014-01-03T02:00:00-05:00",
 "measurement": 34.948730469
 }, {
 "timestamp": "2014-01-03T02:30:00-05:00",
 "measurement": 0.0
 }, {
 "timestamp": "2014-01-03T03:00:00-05:00",
 "measurement": 45.434570313
 }, {
 "timestamp": "2014-01-03T03:30:00-05:00",
 "measurement": 0.390625
 }, {
 "timestamp": "2014-01-03T04:00:00-05:00",
 "measurement": 63.964843751000004
 }, {
 "timestamp": "2014-01-03T04:30:00-05:00",
 "measurement": 15.719449448999999
 }, {
 "timestamp": "2014-01-03T05:00:00-05:00",
 "measurement": 7.025024703
 }, {
 "timestamp": "2014-01-03T05:30:00-05:00",
 "measurement": 0.0
 }, {
 "timestamp": "2014-01-03T06:00:00-05:00",
 "measurement": 26.245117188000002
 }, {
 "timestamp": "2014-01-03T06:30:00-05:00",
 "measurement": 13.256835939
 }, {
 "timestamp": "2014-01-03T07:00:00-05:00",
 "measurement": 19.042968751
 }, {
 "timestamp": "2014-01-03T07:30:00-05:00",
 "measurement": 11152.111816408
 }, {
 "timestamp": "2014-01-03T08:00:00-05:00",
 "measurement": 5815.869140628001
 }, {
 "timestamp": "2014-01-03T08:30:00-05:00",
 "measurement": 151.17821951599998
 }, {
 "timestamp": "2014-01-03T09:00:00-05:00",
 "measurement": 25.939941407
 }, {
 "timestamp": "2014-01-03T09:30:00-05:00",
 "measurement": 0.0
 }, {
 "timestamp": "2014-01-03T10:00:00-05:00",
 "measurement": 2376.452636721
 }, {
 "timestamp": "2014-01-03T10:30:00-05:00",
 "measurement": 4526.770019533001
 }, {
 "timestamp": "2014-01-03T11:00:00-05:00",
 "measurement": 75.08544921999999
 }, {
 "timestamp": "2014-01-03T11:30:00-05:00",
 "measurement": 67.87217916700001
 }, {
 "timestamp": "2014-01-03T12:00:00-05:00",
 "measurement": 0.366210938
 }, {
 "timestamp": "2014-01-03T12:30:00-05:00",
 "measurement": 3.784179688
 }, {
 "timestamp": "2014-01-03T13:00:00-05:00",
 "measurement": 43.823242188
 }, {
 "timestamp": "2014-01-03T13:30:00-05:00",
 "measurement": 9.887695313
 }, {
 "timestamp": "2014-01-03T14:00:00-05:00",
 "measurement": 0.0
 }, {
 "timestamp": "2014-01-03T14:30:00-05:00",
 "measurement": 43.334960937999995
 }, {
 "timestamp": "2014-01-03T15:00:00-05:00",
 "measurement": 6.884765626
 }, {
 "timestamp": "2014-01-03T15:30:00-05:00",
 "measurement": 93.981933596
 }, {
 "timestamp": "2014-01-03T16:00:00-05:00",
 "measurement": 37.866210937999995
 }, {
 "timestamp": "2014-01-03T16:30:00-05:00",
 "measurement": 34.631347657
 }, {
 "timestamp": "2014-01-03T17:00:00-05:00",
 "measurement": 11.547851563
 }, {
 "timestamp": "2014-01-03T17:30:00-05:00",
 "measurement": 7009.921468534001
 }, {
 "timestamp": "2014-01-03T18:00:00-05:00",
 "measurement": 0
 }, {
 "timestamp": "2014-01-03T18:30:00-05:00",
 "measurement": 15.417480469000001
 }, {
 "timestamp": "2014-01-03T19:00:00-05:00",
 "measurement": 252.099609377
 }, {
 "timestamp": "2014-01-03T19:30:00-05:00",
 "measurement": 446.12486734699996
 }, {
 "timestamp": "2014-01-03T20:00:00-05:00",
 "measurement": 1.721191406
 }, {
 "timestamp": "2014-01-03T20:30:00-05:00",
 "measurement": 5.083799915
 }, {
 "timestamp": "2014-01-03T21:00:00-05:00",
 "measurement": 8.203125001
 }, {
 "timestamp": "2014-01-03T21:30:00-05:00",
 "measurement": 6402.282714841999
 }, {
 "timestamp": "2014-01-03T22:00:00-05:00",
 "measurement": 1084.9121093790002
 }, {
 "timestamp": "2014-01-03T22:30:00-05:00",
 "measurement": 0.0
 }, {
 "timestamp": "2014-01-03T23:00:00-05:00",
 "measurement": 29.750045897
 }, {
 "timestamp": "2014-01-03T23:30:00-05:00",
 "measurement": 2.560905829
 }, {
 "timestamp": "2014-01-04T00:00:00-05:00",
 "measurement": 0.0
 }],
 "activity_goal": {
 "minutes": 90,
 "started_at": "2014-01-01T00:00:00-05:00 America/New_York"
 }

 Getting trend data

Request

https://app.whistle.com/api/dogs/9999/trends?date=2014-01-13
GET /api/dogs/9999/trends?date=2014-01-13 HTTP/1.1
Host: app.whistle.com
User-Agent: WhistleApp/102 (iPhone; iOS 7.0.4; Scale/2.00)
Proxy-Connection: keep-alive
Accept: */*
Accept-Encoding: gzip, deflate
Content-Type: application/json
Accept-Language: en;q=1, fr;q=0.9, de;q=0.8, ja;q=0.7, nl;q=0.6, it;q=0.5
Connection: keep-alive
X-Whistle-AuthToken: your_token_that_was_returned_in_the_login_response

Response (JSON)

{
 "days": {
 "2014-01-13T00:00:00-05:00": {
 "total_minutes_active": 115,
 "total_minutes_rest": 1131,
 "activity_goal": 90,
 "average_minutes_active": 124.666666666667,
 "average_minutes_rest": 1090.66666666667
 },
 "2014-01-14T00:00:00-05:00": {
 "total_minutes_active": 103,
 "total_minutes_rest": 1141,
 "activity_goal": 90,
 "average_minutes_active": 107.333333333333,
 "average_minutes_rest": 1084.33333333333
 },
 "2014-01-15T00:00:00-05:00": {
 "total_minutes_active": 101,
 "total_minutes_rest": 1155,
 "activity_goal": 90,
 "average_minutes_active": 112.333333333333,
 "average_minutes_rest": 1066.66666666667
 },
 "2014-01-16T00:00:00-05:00": {
 "total_minutes_active": 125,
 "total_minutes_rest": 1075,
 "activity_goal": 90,
 "average_minutes_active": 121.333333333333,
 "average_minutes_rest": 1102.66666666667
 },
 "2014-01-17T00:00:00-05:00": {
 "total_minutes_active": null,
 "total_minutes_rest": null,
 "activity_goal": 90,
 "average_minutes_active": 105.5,
 "average_minutes_rest": 1060.0
 },
 "2014-01-18T00:00:00-05:00": {
 "total_minutes_active": null,
 "total_minutes_rest": null,
 "activity_goal": 90,
 "average_minutes_active": 73.3333333333333,
 "average_minutes_rest": 802.0
 },
 "2014-01-19T00:00:00-05:00": {
 "total_minutes_active": null,
 "total_minutes_rest": null,
 "activity_goal": 90,
 "average_minutes_active": 103.333333333333,
 "average_minutes_rest": 1095.66666666667
 }
 },
 "number_days_goal_reached": 4,
 "overall_activity_percent_change": 3.84,
 "overall_rest_percent_change": 7.98,
 "similar_dogs_minutes_rest": 1014.8785460992899,
 "similar_dogs_minutes_active": 43.8461879432624,
 "similar_dogs_rest_percent_difference": 10.9,
 "similar_dogs_activity_percent_difference": 153.16,
 "historical_average_minutes_rest": 1042.3,
 "historical_average_minutes_active": 106.9,
 "current_average_minutes_rest": 1125.5,
 "current_average_minutes_active": 111.0,
 "average_minutes_rest": 1042.3,
 "average_minutes_active": 106.9,
 "total_minutes_rest": 1125.5,
 "total_minutes_active": 111.0,
 "next_week_valid": false,
 "previous_week_valid": true
}

It looks like there are some more minor API calls, like setting the activity goal. If I end up persuing this further, I’ll add those later.