Single Page Responsive Web Apps using JavaScript , jQuery and Bootstrap

The ThingSpeak Web Page does not allow you to pull historical data from your channel, plus simple things like converting Absolute Pressure to Mean Sea Level Pressure require running Matlab Scripts.

I decided to create a single page web app that had the features I required. I ended up with the following features:

  • Cross platform/browser
  • Responsive (usable on my 24″ monitor, tablet and phone)
  • Customizable graphs (color, type etc)
  • Ability to select multiple days/weeks
  • Ability to select last minute/hour/day
  • Sea Level Pressure Conversion Built in
  • Auto summation for rain graphs by interval
  • Auto update graphs to get latest data

The web app needs to request data from api.thingspeak.com which is a different domain to the domain where the web app is hosted. To do this I created a route in Ktor which accepts a JSON request and responds with the contents of the request.

post("/getJSON") {
            val thisReq = call.receive()
            val req = khttp.get(thisReq.url)
            println(thisReq.url)
            if (req.statusCode == 200) {
                call.respond(req.jsonObject)
            } else {
                call.respond(json { "success" to "false" })
            }
        }

I understand this could be potentially abused as this is a effectively an anonymous router, but I am not worried just yet.

To access this in JavaScript:

var thingSpeakQuery = `https://api.thingspeak.com/channels/${feedID}/feeds/last.json`;
var getJSONThingSpeak = {
	"url": thingSpeakQuery
};
$.ajax({
	type: "POST",
	url: "/getJSON",
	data: JSON.stringify(getJSONThingSpeak),
	contentType: "application/json; charset=utf-8",
	dataType: "json",
	success: function (element) {
		// code here to handle data
	},
	failure: function (errMsg) {
		alert(errMsg);
	},
	// catch internal server error, this happens when the server can't pass the JSON data,
        // or the ThingSpeak ID is invalid
	statusCode: {
		500: function () {
			console.log("Query returned no data.");
		}
	}
});

Here I am using an AJAX function to get the latest data from the specified ThingSpeakID.

The rest of the web is pretty much standard JavaScript and relies heavily on jQuery to update the web page. The JSON data structures get pretty complicated. In order to debug/test the code while I was writing it I made the returned JSON data a global variable and used Chromium’s Dev Tools to write the code.

For example here where are looping through all the Charts and adding the ThingSpeak data:

thisData.map.feeds.myArrayList.forEach(function (element) {
                // loop over each data field in ThingSpeak data
                for (var k = 0; k < chartList.length; k++) {
                    // store as a tmp string, as not to change with 'k'
                    var tmp = k + 1;
                    weatherCharts[k].data.datasets[0].data.push({
                        x: moment(element.map.created_at),
                        y: parseFloat(element.map["field" + tmp.toString()])
                    });
// snipped

The above would be very difficult to write by hand, or get working by trial and error. Being able to run the code in the browser and make things instantly change was actually a lot of fun.

JavaScript is fast becoming my favorite language, not due to it’s technical merits but due to the fact it allows we to be vendor agnostic. I can run my code on my FreeBSD/Linux/Android/Windows/Mac/iPhone hardware.

The coolest feature, is the auto update feature. It allows me to leave the browser window open on my computer and have a live stream graph of my weather station without having to continuously hit the refresh button!

It’s pretty simple, it just uses a JavaScript interval function:

setInterval(function () {
    // This will be executed every 60 seconds
    getLatestWeatherForWeatherCharts();
    }, 60000);

You can use the web app here.

opens3.net_weatherstats.html

Simple Content Management System using Kotlin/JQuery/Ktor/MySQL

I wrote a simple Content Management System using JQuery, Kotlin and Ktor. I did this to learn more about JavaScript, MySQL and the webstack before moving on to learn a web framework (React).

There was a fair bit of JS code, and several routes that had to be created, but it wasn’t that difficult and only took two days.

The current system runs on my website (opens3.net) and has a front end and backend that run as separate services.

You can view the source code here and here.

fireshot20capture2000120-20opens320admin20dashboard20-20https___opens3.net_5030_

Weather Bot goes live

I re-wrote my weather bot completely in Kotlin and split the functionality into three parts:

  1. The back-end downloads RSS feeds and compares them against the user database, it sends the alerts to user’s mobiles
  2. The front-end adds users and handles verification
  3. The front-end administration panel allows for addition, deletion of users.

Both the front-ends are written using jQuery, Bootstrap, vanilla Javascript and Kotlin/ktor. They are both single page web apps and allow for all functions without page reload.

Here is the sign up page on my website:

screencapture-opens3-net-main-html-2019-01-22-07_59_04

Here is the admin interface:

alerts

The database functions are handled using data classes and SQL objects using a SQL DSL for Kotlin caused Exposed.

Below is an example of the object classes, data classes and a function required to return a list of user objects. This code is used to return a list of users to the admin interface.

The Kotlin SQL DSL makes the programming of very complicated data structures a breeze.

object users : Table() {
    val id = integer("id").autoIncrement().primaryKey() // Column
    val name = varchar("name", length = 150) // Column
    val country = varchar("country", length = 150) // Column
    val state = varchar("state", length = 150) // Column
    val mobile = varchar("mobile", length = 50) // Column
    val opt1 = varchar("opt1", length = 50).nullable() // Column
    val opt2 = varchar("opt2", length = 50).nullable() // Column

}

object userkeywords : Table() {
    val id = integer("id").autoIncrement().primaryKey() // Column
    val uuid = integer("uuid").nullable()
    val keywords = varchar("keywords", length = 150) // Column
}

data class completeUser(
    val id: Int,
    val name: String,
    val country: String,
    val state: String,
    val mobile: String,
    val keywords: MutableList,
    val opt1: String,
    val opt2: String
)
fun getAllUsers(): MutableList {
    connectToDB()
    var returnedUsers = mutableListOf()
    transaction {
        SchemaUtils.create(users, userkeywords)
        val allKeywords = userkeywords.selectAll()
        for (user in users.selectAll()) {
            val thisUser = completeUser(
                id = user[users.id],
                name = user[users.name],
                country = user[users.country],
                state = user[users.state],
                mobile = user[users.mobile],
                opt1 = "NIL",
                opt2 = "NIL",
                keywords = mutableListOf()
            )
            println("${user[users.id]}: ${user[users.name]}")
            for (k in allKeywords) {
                if (k[userkeywords.uuid] == user[users.id]) {
                    val kw = k[userkeywords.keywords]
                    thisUser.keywords.add(kw)
                }
            }
            returnedUsers.add(thisUser)
        }

    }
    return returnedUsers
}

To send this data in JSON it is very easy.

First “install” the GSON feature:

install(ContentNegotiation) {
        gson {
            setDateFormat(DateFormat.LONG)
            setPrettyPrinting()
        }
    }

Then add the route:

get("/getUsers") {
    var theseUsers = getAllUsers()
    call.respond(theseUsers)
}

Pretty braindead stuff really. Handling queries in JavaScript is also very easy (the following function adds all retrieved users to a drop down menu):

var getUserList = function() {
  $.ajax({
    type: "GET",
    url: "/getUsers",
    // The key needs to match your method's input parameter (case-sensitive).
    contentType: "application/json; charset=utf-8",
    dataType: "json",
    success: function(data) {
      $('#deleteUser').empty();
      data.forEach(function(element) {
        $('#deleteUser').append(`
${element.name} mobile: ${element.mobile}`);
      });
    },
    failure: function(errMsg) {
      alert(errMsg);
    }
  });
};

As for the back-end, I had to write the API in Kotlin to connect to the Telstra API. I had to use khttp because ktor client can not set custom headers at the moment.

Here is an example of how it works:

//provision a number
val response = khttp.post(
      url = "https://tapi.telstra.com/v2/messages/provisioning/subscriptions",
      headers = mapOf("content-type" to "application/json","authorization" to "Bearer ${token}" , "cache-control" to "no-cache"),
            data = "{ \"activeDays\": 30 }")

// get/update API token
val payload = mapOf(
      "grant_type" to "client_credentials",
      "scope" to "NSMS",
      "client_id" to "xxx",
      "client_secret" to "xxx")

val response = khttp.post(
      url = " https://tapi.telstra.com/v2/oauth/token",
      headers = mapOf("Content-Type" to "application/x-www-form-urlencoded"),
      data = payload)

// send a message
val response = khttp.post(
      url = " https://tapi.telstra.com/v2/messages/sms",                  
      headers = mapOf("content-type" to "application/json",
      "authorization" to "Bearer ${token}"),
      data = ourJson.toString())
 

I had to setup a dummy route in ktor and use WireShark to sniff the packets to figure out which headers needed to be set before I could finally figure out how the system worked.

Thanks to Telstra for the free 1000 messages per month and thanks to the folks at Jetbrains for all their awesome libraries and products.

Web Apps with Kotlin+Ktor

I rebuilt my website using Kotlin, Ktor, Bootstrap and Vanilla JS. I am happy with the results so far.

I am still working my way through a book on Ecmascript 6 and have to admit that programming in modern JavaScript is quite fun. I am also happy with the Ktor web framework.

2019 is my year of web apps and I hope to create hundreds of them in the future. I feel that web apps are the future and enjoy the flexibility afforded by the web platform.

View the website here. Get the source code here.

Updated Weather Station with OTA and configuration via WiFi AP

I used the great code here: https://github.com/HarringayMakerSpace/usb-esp

To add configuration via WiFi and OTA updates via ThingSpeak to my weather station, I also completely redesigned the base plate incorporating new components I have received from eBay. The new design allows the battery to be accessed as well as the reset switch on the Wemos D1 Mini.

I added the following parts:

I am no longer using the Wemos D1 Mini Battery Shield. See my GitHub repository for details, go to the my Thingiverse page to download the files, go to the ThingSpeak live feed to see live data.

 

 

Creating decks programmatically Kotlin/Perl for Anki/AnkiDroid

I used Google’s Text To Speech API to create mp3 files for each of the top 6000 Korean words, I then created a csv file using Perl and imported them into Anki.

The CSV file is imported as a cloze deletion card with the audio files as the extra field.

The code is here: https://github.com/wilyarti/anki-translator

This year I am focusing on learning Kotlin/C/JS and this is the first Kotlin script/program. I am finding Kotlin an easy transition from my mostly Go/Perl background.