Setting up quick API with F# and Azure Functions

As mentioned in my last week Elixir blog post, I produced some quick fake API based on Azure Functions. I thought it’s gonna take a couple of minutes, but it turned out to be a whole adventure in itself.

The creating of a function is a breeze.

  1. Go to Portal, click big green “+” sign and search for “Function App”Screen Shot 2017-04-17 at 16.34.00.png
  2. Pick Function App published by Microsoft
  3. Fill all the necessary fields like App name (must be globally unique) or location. For hosting plan I used “Consumption plan” which means, I pay only for the time that function is running. I also like to pin my stuff to the dashboard, so it’s easier to find.Screen Shot 2017-04-17 at 16.37.38.png
  4. It will take several minutes to deploy.
  5. Now you can create your functions for the app. F# is hidden in small print just above “Create this function” button. So click “create your own custom function”.Screen Shot 2017-04-17 at 16.40.40.png
  6. Then with Language drop-down, pick “F#” and for Scenario – “API  & Webhooks”. There should be on the F# function triggered by HTTP request. That’s the one you want for API.
  7. You’ll get premade piece of code with a simple function that is triggered by HTTP POST with name object and responses “Hello “.

Then I started writing the logic I wanted. I made an array of hard coded airport data. I made the function to accept only GET requests (you can change it in function.json file). In code, I parse query strings and get the airport IATA code. If I have this airport in my array, I response 200 with JSON containing the data. Otherwise, I return 404. If there’s no parameter in the query string, function answers with 500.

It’s relatively simple and straightforward F# code. I just struggled a lot with debugging. The small editor on Azure doesn’t give you static analysis, nor type information and no squigglies. You need to run the function and check for compilation errors or runtime errors. There was also some weird scoping behaviour, that forced me to declare the Airports array within the function. Anyways, after 2hrs I had an API that did what I wanted. You can see the code below. It’s not bulletproof, but it does the job. And I got to play with Azure Functions a bit.

If you want to read more about other types of F# Azure Functions, Mathias Brandewinder wrote recently two posts about timer and queue triggered functions.

That’s all for today. Tune in next week for another part. Also, check previous episodesAnd if you’re interested in machine learning, look into my weekly link drop.

Advertisements

Calling an external API from Phoenix app

Last week I wrote about how to extend your Ecto model. I wanted to figure it out because I thought this week, I’ll add to mother other flight related data. Turns out, I did it in completely different way. Not sure if this is the correct approach, so feel free to bash mi in comments (in a constructive way!).

One of the data I have on my flights so far is a departure and destination airport IATA code. Those are 3 letters code, commonly used in civil aviation (SFO = San Francisco, CPH = Copenhagen etc.).  I wanted to use some simple API to get more data based on this code. So I decided to write my own using Azure Functions and F#. It became a separate adventure. I’ll write another post about it.

When I had my API ready, I started figuring out, how to put it in Phoenix app. My first approach was a bit like you do stuff in ASP.NET MVC – adding those extra fields to the model, and trying to update them from within the controller. Unfortunately, I couldn’t make it work. I couldn’t even access fields of the model.

The solution I ended up doing, is putting all the web calls and logic for parsing them within the view file. It feels dirty for me. It would be very dirty in ASP.NET MVC. So my intuition is, that it’s not the best way to do it. But so far the only one I made working. After those changes my Flight view file looks like that:

And then I use those functions within the template that renders the view:

Immediately you can see another problem with the code – for every information that I want to show, I’m making a separate web request. But it works, which is a huge improvement. Done is better than perfect.

Screenshot to prove that it does, what it should:

Screen Shot 2017-04-16 at 01.52.21.png

 

That’s all for today. Tune in next week for another part. Also, check previous episodesAnd if you’re interested in machine learning, look into my weekly link drop.

Weekly ML drop #7

I’ve become more and more interested in machine learning during last year. This is my way of collecting and sharing interesting reads on the topic I stumble upon. Those posts are published each Friday, they are divided into few categories and the format is constantly evolving.

News
In this part, I share interesting news from machine learning and artificial intelligence world. Those are mostly not very scientific articles about interesting applications,  predictions and controversies that AI causes.

Exploring the mysteries of Go with AlphaGo and China’s top players
AlphaGo together with Chinese Go Association is bringing together the most talented Go players and computer scientist to explore deeper into the game. Turns out, that last year victory of AlphaGo over human player totally changed how humans play this game.

Who will pay insurance in the era of self-driving cars?
This article discusses impact self-driving cars may have on the insurance industry. As it is suggested currently by analysts, and some first legislation follows, car makers may be responsible for that cost.

Why deep learning is suddenly changing your life?
Just another article that discusses what exactly happened in recent years, that machine learning seems to be everywhere and changing everything. It also explains basic terms and methods used in the field.

AI based hedge fund created a new currency
The article, with a bit sensationalist title, discuss new fintech startup Numerai, that uses open market for AI algorithms to make its trading decisions and rewarding authors of the best algorithms.

How predictive AI will change shopping
The author of this texts brings up few examples how AI revolution impacts the retail world. Thanks to better data collections, connected devices and putting this data together, retail companies can offer better suggestions, increasing their profits.


Learning materials

Here I’m sharing material for learning ML that I found useful – online courses, blogs, books etc. This is usually rather technical stuff.

Artificial Intelligence and the Growing Importance of Soft Skills
Very interesting read about which skills are endangered and which not by AI. If you want to prepare for robots taking your job, that’s a good start.

Introduction to Machine Learning
This short video from Intel Nervana AI academy explains basic terms of Machine Learning and most popular types of it.

Stanford series on Natural Language Processing with Deep Learning
Another academic source of knowledge. I haven’t had time to dig into it, but quick skimming suggests, that’s another amazing 20hrs of the content of ML knowledge, this time targeted at NLP.


This is it for today, thanks for reading. If you liked the post, let me know and please check other parts of the series.

Extending your Ecto model

Last week I used Ecto models to quickly created the database and I was very surprised how all got generated for me. But the model I build was very simplistic, and now I need to extend it. I started working on first serious functionality for the project, and I’ll have to do some changes. For start, I’m just testing by adding one field.

I generated new migrations file in /priv/repo/migrations folder and a bit by trial and error I end up with file like that:

defmodule Flightlog.Repo.Migrations.CreateFlight do
  use Ecto.Migration

  def change do
    alter table(:flights) do
      add :plane_type, :string
    end

  end
end

And after running mix ecto.migrate, I actually got some results:

Michals-MBP:flightlog michal$ mix ecto.migrate
01:01:37.663 [info]  == Running Flightlog.Repo.Migrations.CreateFlight.change/0 forward
01:01:37.663 [info]  alter table flights
01:01:37.666 [info]  == Migrated in 0.0s

This worked for adding fields to the database but didn’t automagically update all the access layers. There’s probably some way to it, but this time I did it manually.

First I updated the views, for example added following into show.html.eex:

This solved the visuals but still didn’t work. The crucial were changes in the model:

  defmodule Flightlog.Flight do
  use Flightlog.Web, :model

  schema "flights" do
    field :date, Ecto.DateTime
    field :flight_number, :string
    field :plane_type, :string
    field :from, :string
    field :to, :string

    timestamps()
  end

  @doc """
  Builds a changeset based on the `struct` and `params`.
  """
  def changeset(struct, params \\ %{}) do
    struct
    |> cast(params, [:date, :flight_number, :plane_type, :from, :to])
    |> validate_required([:date, :flight_number, :plane_type, :from, :to])
  end
end

I added new field both in schema part and in changeset. Cast is responsible for things being updated. I didn’t have to add it to be validates as required.

So this was actually a bit tedious, but I’m probably missing something here. Hopefully I’ll figure it out by the time I’ll need to make bigger changes.

I also found this post, that’s deal with similar problem.

That’s all for today. Tune in next week for another part. Also, check previous episodesAnd if you’re interested in machine learning, look into my weekly link drop.

Weekly ML drop #6

I’ve become more and more interested in machine learning during last year. This is my way of collecting and sharing interesting reads on the topic I stumble upon. Those posts are published each Friday, they are divided into few categories and the format is constantly evolving.

Accidentally this week, I also have a theme. Most articles are related to impact on workplaces and how companies work.

News
In this part, I share interesting news from machine learning and artificial intelligence world. Those are mostly not very scientific articles about interesting applications,  predictions and controversies that AI causes.

Axa is using ML to predict big car accident with 78% accuracy
In this article on Google Platform blog, the case of a big insurance company is discussed. What technology was used and how it worked.

How automation impacts economy?
This post discusses potential outcomes of more and more automation in modern economies.

Artificial intelligence could dramatically improve the economy and aspects of everyday life, but we need to invent ways to make sure everyone benefits.

How AI is transforming workplace
Discussion of multiple aspects of workplace and how machine learning and other AI-related technologies may change them.

How AI is changing the way companies are organised
This article brings up some examples how introducing ml tools may change how companies are structured and how internal communication is performed.

Machine learning used to smart compression of videos
Netflix uses new machine learning based algorithm to compress video scene by scene for better results.

Opinions of 17 experts, how worried we should be about AI


Learning materials

Here I’m sharing material for learning ML that I found useful – online courses, blogs, books etc. This is usually rather technical stuff.

In-depth, non-technical guide to machine learning
This five-part article goes through terms and techniques used in machine learning in human-friendly language. Very good first contact with anything machine-learning related.


This is it for today, thanks for reading. If you liked the post, let me know and please check other parts of the series.

Connecting to database with Ecto

In the last episode of my Elixir adventures, I messed with own-made Views and Controller to display hardcoded flight information for my FlightLog. This week I’ll try to show data based on the content of the database.

Phoenix doesn’t have built-in data access capabilities. But there’s awesome Ecto project, that’s sort of beefed up ORM. It reminds me Entity Framework (or good parts of it) in .NET or Rails. It supports multiple databases, although the default is Postgres. If you generated your Phoenix project by default and didn’t exclude Ecto, you should have everything you need to start. If no, refer to this guide

To add data access to FlightLog, I started the way, I would usually do in a web project. I installed the database (I went with Postgres), created some table, inserted some sample data. It was proven later, that it wasn’t a necessary step.

To build your first model, go to your project root and type for example:

$ mix phoenix.gen.html Flight flights date:datetime flight_number:string from:string to:string
* creating priv/repo/migrations/20150409213440_create_flight.exs
* creating web/models/flight.ex
* creating test/models/flight_test.exs
* creating web/controllers/flight_controller.ex
* creating web/templates/flight/edit.html.eex
* creating web/templates/flight/form.html.eex
* creating web/templates/flight/index.html.eex
* creating web/templates/flight/new.html.eex
* creating web/templates/flight/show.html.eex
* creating web/views/flight_view.ex
* creating test/controllers/flight_controller_test.exs

Add the resource to your browser scope in web/router.ex:

    resources "/flights", FlightController

and then update your repository by running migrations:

    $ mix ecto.migrate

As you can see, this gave me a lot of stuff – a migration, a controller, a controller test, a model, a model test, a view, and a number of templates. It also instructs as to add new Controller to the router. Let’s do that.

I also removed recent additions, because they served the same purpose. I initially though I’ll just edit them, but Ecto surprised me by doing everything for me. Updated route file looks like that:

defmodule Flightlog.Router do
  use Flightlog.Web, :router

  pipeline :browser do
    plug :accepts, ["html"]
    plug :fetch_session
    plug :fetch_flash
    plug :protect_from_forgery
    plug :put_secure_browser_headers
  end

  pipeline :api do
    plug :accepts, ["json"]
  end

  scope "/", Flightlog do
    pipe_through :browser # Use the default browser stack

    resources "/flights", FlightController
  end

  # Other scopes may use custom stacks.
  # scope "/api", Flightlog do
  #   pipe_through :api
  # end
end


After that I run mix ecto.migrate which executed migration file:

defmodule Flightlog.Repo.Migrations.CreateFlight do
  use Ecto.Migration

  def change do
    create table(:flights) do
      add :date, :datetime
      add :flight_number, :string
      add :from, :string
      add :to, :string

      timestamps()
    end

  end
end


This created tables for me as defined in phoenix.gen.html. I noticed that database has some extra fields for basic journaling like inserted_at and updated_at. I logged into the database and added some data just to have something to show and fired up my project again:

mix phoenix.server

When you browse to the site, you’ll notice that not only I have a list of flights, but also buttons for actions like Edit, Delete or Add a new item. All generated for me and even with decent styling. That was a very nice surprise.

Screen Shot 2017-04-06 at 13.00.42.png

So first look at Ecto is very positively surprising. It created all the boilerplate code for me, but it is also a code that’s understandable and easy to edit. There seem not to be any underlying magic. If I need to change default behaviour I feel I wouldn’t have much problem doing that. We’ll see in future if practice will support those claims.

I haven’t covered any of the internals how Ecto works, so if you’re interested I suggest you peek into this guide and official documentation.

That’s all for today. Tune in next week for another part. Also, check previous episodesAnd if you’re interested in machine learning, look into my weekly link drop.

Weekly ML drop #5

I’ve become more and more interested in machine learning during last year. This is my way of collecting and sharing interesting reads on the topic I stumble upon. Those posts are published each Friday, they are divided into few categories and the format is constantly evolving.

This week I gathered a lot of articles on the topic of AI for self-driving cars and this will be a theme for this edition.

News
In this part, I share interesting news from machine learning and artificial intelligence world. Those are mostly not very scientific articles about interesting applications,  predictions and controversies that AI causes.

Artificial Intelligence will enable jobs too
Usually, in the context of AI, you can hear predictions of people losing jobs. In this story about IBM “new collar” jobs, we can read that company is aiming at hiring people uneducated in cyber security who have necessary set of natural skills, and augment them with technology to fill lackings in professional skills.

Intel forms new AI division
Intel’s been behind Nvidia in developments of processing units that power AI revolution. Recent acquisitions mentioned last week and forming new group shows that they’re not gonna give up easily. In a similar move, YCombinator is gonna run experimental vertical group focused on AI in next batch of founded startups.

Andrew Ng leaves Baidu
He’ll now focus on “shepherding this important societal change”. He also wants to support ML community around the world. I love the quote from his blog.

Just as electricity transformed many industries roughly 100 years ago, AI will also now change nearly every major industry — healthcare, transportation, entertainment, manufacturing — enriching the lives of countless people. I am more excited than ever about where AI can take us. — Andrew Ng

Agents in OpenAI research develop their own language
In this blog post, OpenAi discusses some results from the research that aimed at teaching AIs developing a new language of communication “by dropping them into a set of simple words, giving them the ability to communicate, and then giving them goals that can be best achieved by communicating with other agents”.

Learning materials
Here I’m sharing material for learning ML that I found useful – online courses, blogs, books etc. This is usually rather technical stuff.

Algorithmia’s not very heavy intro to Deep Learning
This is not very heavy article explaining on a high level what Deep Learning is.

Convolutional Neural Networks
Recently in my learnings, I was exploring the topic of CNNs. This is a type of neural networks that are very good at image recognition tasks. This series of articles going through interesting papers on the field and this papers were especially interesting. I highly recommend them if you’re interested in the topic.

This is it for today, thanks for reading. If you liked the post, let me know and please check other parts of the series.