Wednesday, June 01, 2016

Converting unix epoch to datetime in streaming analytics

sample input:
    {
        "Key": "Dryer_Sensor",
        "State": "0.8",
        "Timestamp": "1464782405968",
        "Site": "Bir57",
        "EventProcessedUtcTime": "2016-06-01T13:24:49.4517020Z",
        "PartitionId": 0,
        "EventEnqueuedUtcTime": "2016-06-01T12:00:06.4520000Z",
        "IoTHub": {
            "MessageId": "7e6d0379d85046f295c0cdaeaabe25d8",
            "CorrelationId": null,
            "ConnectionDeviceId": "openhab",
            "ConnectionDeviceGenerationId": "635957210185596241",
            "EnqueuedTime": "0001-01-01T00:00:00.0000000",
            "StreamId": null
        }
   }

 query: DATEADD(millisecond, CAST([Timestamp] as bigint), '1970-01-01T00:00:00Z') as timeFromString

this will result in  '1970-01-03T07:09:18.032Z' which is wrong!

you can correct this by modifying the input so its strips the quotes from the epoch


    {
        "Key": "Dryer_Sensor",
        "State": "0.8",
        "Timestamp": "1464782405968",
        "epochtime": 1464782405968,
        "Site": "Bir57",
        "EventProcessedUtcTime": "2016-06-01T13:24:49.4517020Z",
        "PartitionId": 0,
        "EventEnqueuedUtcTime": "2016-06-01T12:00:06.4520000Z",
        "IoTHub": {
            "MessageId": "7e6d0379d85046f295c0cdaeaabe25d8",
            "CorrelationId": null,
            "ConnectionDeviceId": "openhab",
            "ConnectionDeviceGenerationId": "635957210185596241",
            "EnqueuedTime": "0001-01-01T00:00:00.0000000",
            "StreamId": null
        }
   }

and modifying the query so it omits the cast (where the bug resides):  DATEADD(millisecond, epochtime, '1970-01-01T00:00:00Z') as time

this will result in the expected result: 2016-06-01T12:00:05.968Z


Friday, April 08, 2016

Registering a device in Azure IoT Hub


Using Node.js

npm install -g iothub-explorer@latest
iothub-explorer <connection string> create MyDevice

source

Using Device Explorer

See: azure iot sdk on github

Friday, February 26, 2016

Installing Z-Wave Stick Gen 5 on Windows IoT Core


I am currently fiddling around with Windows 10 IoT Core on a raspberry Pi. I wanted to use Z-Wave to be able to control a Fibaro Wall Plug (http://www.fibaro.com/uk/the-fibaro-system/wall-plug)

But I could not find the Z-Stick that was on the supported hardware list of windows iot core

so I bought a Z-Wave Aeon Labs Z-Stick USB Controller - Gen5.

the following steps will explain how to get it working for windows iot core

Step 1
Download the drivers here: http://aeotec.com/z-wave-usb-stick/1358-z-wave-drivers.html

step 2
Connect to tou you raspberry e.g. \\freyr\c$\Data\Users\Administrator\Documents
and unzip and copy the drivers

step 3
connect using powershell

[freyr]: PS C:\Data\Users\Administrator\Documents>

freyr]: PS C:\Data\Users\Administrator\Documents> dir


    Directory: C:\Data\Users\Administrator\Documents


Mode                LastWriteTime         Length Name
----                -------------         ------ ----
-a----        8/24/2015   8:34 PM           8424 uzb.cat
-a----        8/24/2015   8:34 PM            710 uzb.inf

[freyr]: PS C:\Data\Users\Administrator\Documents> devcon dp_add .\uzb.inf

....

Driver package 'oem0.inf' added.
[freyr]: PS C:\Data\Users\Administrator\Documents>

[freyr]: PS C:\Data\Users\Administrator\Documents> devcon status usb*

...

USB\VID_0658&PID_0200\5&3753427A&0&4
    Name: USB Serial Device
    Driver is running.

...

Thursday, March 05, 2015

About CQS, CQRS and Event Sourcing

Seems to be a lot of confusion going on about these terms. The below extract form Greg Young explains it all:
Many people have been getting confused over what CQRS is. They look at CQRS as being an architecture; it is not. CQRS is a very simple pattern that enables many opportunities for architecture that may otherwise not exist. CQRS is not eventual consistency, it is not eventing, it is not messaging, it is not having separated models for reading and writing, nor is it using event sourcing. I want to take a few paragraphs to describe first exactly what CQRS is and then how it relates to other patterns.

Reference: http://codebetter.com/gregyoung/2010/02/16/cqrs-task-based-uis-event-sourcing-agh/


Thursday, May 15, 2014

What is the difference between IaaS, Paas and SaaS?

Software as a Service (SaaS)

This has been around as long as the internet exists. These are pieces of software that you can subscribe to, like for instance Gmail, outlook.com, office 365 but as well as Adobe's creative cloud for instance. The main characteristics of this model is that you always are up to date with the latest version. If you take the example of outlook.com (Hotmail), Microsoft will not ask you to update it, you cannot rollback to a previous version. This off course will require from the provider that they always have to ensure continuity of their service. Since I as a customer would not appreciate it that I would not be able to read the emails in my inbox I received or sent 5 years ago.


Platform as a Service (PaaS)

In Microsoft azure these are all the features Microsoft makes available to you the only thing you need to worry about is get your code on there and Microsoft will keep all these features up to date to the latest versions. You have little or no Control over the Operation System or the underlying Infrastructure. In return off course you get certain things in return. For instance in Azure Websites You can have zero downtime scaling, high availability, and failover. Cloud Services provide you with more control in configuring the underlying infrastructure but that's it, you cannot install additional components on these features like a SQL server, you get an OS, an IIS and that's it, there is however some control in the version of the underlying OS and network (expose additional endpoints, deploy on a virtual network, etc). 


Infrastructure as a Service (IaaS)

These are basically virtual machine, the customer is responsible to monitor and maintain the OS and install and maintain the necessary features on these Virtual Machines like e.g. IIS, MSMQ, …
It's all matter of what you want to do yourself, are willing to do yourself



Scott Hanselman provides a very good analogy with regards to this: http://www.hanselman.com/blog/IntroducingWindowsAzureWebJobs.aspx

Tuesday, April 29, 2014

Microsoft Azure VPN: Error 0x8007274c whilst trying to connect

When trying to connect you are getting the below error message:

A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond.
 (Error 0x8007274c)

try to upload the root certificate again.

Wednesday, April 23, 2014

Microsoft Azure: Reducing Cost

If you plan to use IaaS make sure you have a sound monitoring strategy that you have reporting on the load and performance of your Virtual Machines.



 

As the above graph shows that the virtual machine is not used during the last 7 days. This is a very good candidate to look a little bit deeper into. After doing some root cause analysis it is a standard small vm running linux costing your company around € 33.25 / a month. If we would scale down this application from Standard to Basic we would save € 5.54 / month which would be a cost reduction of ~17%. Since it seems that this virtual machine is underutilized for most of the time (CPU anyways) we could scale it down from S to XS bring down the cost even further from € 27.71 to € 9.98 which would save us an additional 64 %. Doing this it would mean that we would bring down out monthly cost from €33.25 to € 9.98 saving us 70%.
If we would take it a step further we could investigate the type of application that is running on this serer and maybe migrate it to a PaaS solution bringing the cost even further down. In this case the VM was running a very simple website hosting this in a shared website we could even bring the cost further down to € 7.21 which would our bring our total savings to 92%.

The cloud is not only about up scaling it is all about optimizing cost! This means that if needed you can scale up and or out but it means you can scale down and in as well saving your company money. This is a very very simple example. But if you would closely analyze usage and optimize your scaling strategy, the potential cost savings can be quite significant. This was a case which I came across in my company, the figures are representative and taken from the Microsoft pricing calculator which can be found at: http://azure.microsoft.com/en-us/pricing/calculator/, not taken into account any EA reductions.

 


Monday, March 24, 2014