Andrew's Blog

Random Thoughts of an ASP.Net Code Monkey

Using the Elasticsearch Azure Plugin with Windows Virtual Machines

February 4, 2014 17:40 by Andrew Westgarth

Recently I’ve been working with Elasticsearch as I look to implement a logging solution using Logstash.  Elasticsearch is a popular, flexible open source real-time search and analytics engine and I’ve been looking at making use of the engine to perform full text searches and analytics against the mountains of log data we have on our platform.

As part of this work I’ve been setting up an Elasticsearch cluster of three Windows Server 2012 R2 Datacenter edition virtual machines in Windows Azure.  We’ve chosen Windows based virtual machines as the team who will support it have more experience and knowledge of Windows OS than Linux.  To get up an running I’ve used the following blog post from Thomas Ardal, which has a further link to info on installing Elasticsearch on the VMS – Running ElasticSearch in a cluster on Azure.

One of the elements of configuring the cluster concerns the discovery of nodes in the cluster.  By default Elasticsearch makes use of multicast to discover the nodes in the cluster, however multicast doesn’t appear to work fully in Azure at present, therefore Thomas’ blog post details making changes to the elasticsearch.yml configuration file to disable multicast and add in the node ip addresses manually. 

However on Friday (31st Jan 2014) Elasticsearch announced an Azure Cloud Plugin for Elasticsearch and the goal for this plugin is to enable automatic unicast discovery of Windows Azure Virtual Machines in the Cloud Service.  The article details how to set the plugin up in Linux Virtual Machines, but as I’ve already mentioned we are using Windows Server 2012 R2 for all of our virtual machines in the cluster.  Therefore I set off today to configure the plugin for running Elasticsearch on Windows.

  1. Create Windows Azure VMs as detailed in Thomas’ post, installing Elasticsearch on each Virtual Machine you wish to use as a node.
  2. Create a self-signed certificate for working with the Azure REST API using makecert (You will need the Windows SDK for this in order to make use of makecert). Create Self Signed Certificate
  3. Upload the .cer file to the Management Certificates (Settings->Management Certificates) store for your Azure subscription in the Azure Portal.Upload Management Certificate 
  4. Using Certificate Manager (certmgr.msc) export the certificate with the public key (in .pfx format) and give the certificate a password of suitable complexity.  Copy the pfx file to a folder on your virtual machines, e.g. C:\MyCertificates
  5. Install the Azure-Cloud plugin on each of your Elasticsearch Virtual Machines by running the following command from a command point at your elasticsearch installation directory
    bin/plugin -install elasticsearch/elasticsearch-cloud-azure/1.0.0.alpha1
    Install Elasticsearch Azure Cloud Plugin
  6. Next edit the Elasticsearch configuration file, Elasticsearch.yml, which is found in the config folder of your Elasticsearch installation on each virtual machine in your cluster.  At the bottom add the following configuration and update with the relevant settings:

    ################################# AZURE PLUGIN ###############################
    cloud:
      azure:
       keystore: <path to certificate i.e. c:/MyCertificates/mycert.pfx>
       password: <Password for certificate file>
       subscription_id: <your subscription id>
       service_name: <name of cloud service>
     
    discovery:
       type: azure

    for example:

    ################################# AZURE PLUGIN ###############################
    cloud:
      azure:
       keystore: c:/MyCertificates/mycert.pfx
       password: verystrongpassword
       subscription_id: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
       service_name: myescluster
     
    discovery:
       type: azure

    Thanks to David Pilato and Thomas Conté for their help in getting the configuration correctly formatted and populated.

  7. Start Elasticsearch on each server

That’s it all configured, if you’re using plugins to monitor your Elasticsearch cluster such as the Head plugin you should now see all of your nodes in your cluster listed, for example:

head

WARNING: As a side note, it’s VERY easy to break the syntax in yml files when not familiar with them, I’m now using Notepad++ to work with them and I’ve also been using a VS Extension – YamlDotNetEditor - when editing the yml file so I can reduce the risk of making mistakes.  The extension formats and lays out the file in a readable format with syntax highlighting (although this is currently not working in VS2013).

Bringing A Code Club to 9-11 Year Olds

January 19, 2014 18:06 by Andrew Westgarth

This is a cross post from my blog on STEMNet. - http://networking.stemnet.org.uk/blog/bringing-code-club-9-11-year-olds

For a while now I’ve been a STEMNet Ambassador working with local schools on Technology related activities and have been working on getting involved in Code Clubs for quite some time.  This week I started my first Code Club with a local school.

logo-8f4e1ca6525af0d4b681e5bea5dbe3d6

I am a strong believer in giving children the opportunity to discover the fun in Computer Science and Computer Programming.  I believe the current National Curriculum can be boring, mundane and uninspiring so much so that we are losing many, many potentially great programmers, developers and technologists at a very early age.  I have worked with a variety of age groups and unfortunately the size of groups get smaller as they get older because they don’t see the fun in the subject, often spending year after year working on nothing more stretching than Microsoft Office (PowerPoint, Excel and Word), admittedly not all schools are this limited but in my experience this has been more a case of the exception rather than the norm.

However, in a rare and unrepeated occurrence, Michael Gove actually said something I agreed with, in that Computer Science is to become an important part in the National Curriculum from September 2014 with children getting the opportunity to explore and experience Computer Science, looking at design, coding and algorithmic skills.  This really struck a chord with me and is something I’ve been keen to get involved with for quite some time.  In order for the UK to encourage students to look further into all scientific subjects, we need to capture the imaginations of students at an early age, i.e. before they leave Primary School (age 11), and then continue to stretch their imaginations and interests throughout Secondary education and onwards hopefully to Further Education or positions in the industry.

Schools are currently not fully equipped with the knowledge, skills and resources to teach coding skills and this presents a great opportunity for our industry to build strong links with the education system, providing valuable support and resources to help bridge the gap.  One such way that I have started to do this, amongst others, is to run a Code Club at a local school.  Code Club is a UK wide network of after school coding clubs for children aged 9-11 run by volunteers.

This past week I held my first Code Club session and what a fantastic and rewarding session it was!  Using the wonderful resources and tutorials prepared by Code Club, I was able to introduce the group of approximately 11 to Scratch, a programming language used to teach children how to build animations, games amongst others.  I started with a quick overview and demonstration of the Scratch IDE and then set the group off with a tutorial with which they were able to build a Cat and Mouse game using sprites and logic they put together using Scratch.  Within minutes of working with the tutorial they were all looking at ways in which they could stretch the possibilities and inject their own customisations into the program.  The hour long club past very quickly and when I had to tell the pupils that it was time to go home and their parents were waiting for them, there was a chorus of disappointed “Awwwws” as they wanted to carry on working.  The pupils seemed to really enjoy it and many took the tutorial notes home so they could continue working through the week until the next Code Club!

If you’re reading this post, work in the IT Industry and have an hour to spare a week why not consider getting involved in helping children to learn to code? Check out the various initiatives in your area and nation, including Code Club, Hour of Code; and look to see if there are any schools or community groups in your area who are looking for a volunteer.  Personally I think it’s the least we can do.  We complain that there aren’t enough good new people for us to hire in our industry, let’s take aim at the long term goal and look to help the next generation of coders grasp the opportunities that our wonderful industry provides!

Book Review: SignalR Real-time Application Development

July 5, 2013 14:34 by Andrew Westgarth

I've been keeping an eye on and experimenting with SignalR for about 18 months now when I get some free time and recently finished reading this book by Einar Ingebrigtsen and published by Packt Publishing.  SignalR: Real-time Application Development is a very concise look at the key topics that you'd want to get up and running with SignalR.

 

The book covers topics such as Security and Scalability along with some simple step by step tutorials to help start using SignalR in a variety of scenarios - straight forward web apps, console apps, WinRT applications amongst others.  At just 124 pages long this book is not exhaustive and is a really useful start for someone who just wants to hit the ground running with SignalR fundamentals.

I found the book a really light read however personally I would have preferred a more detailed book with more discussion of the core technologies employed by SignalR and how they can be exploited to their full potential.  Unfortunately this book focuses on the typical Hello World SignalR application of a chat application a lot of examples of which can be found freely on the internet.

Awarded Microsoft MVP Award for 2013

July 1, 2013 21:12 by Andrew Westgarth

This afternoon I received an email from Microsoft to tell me that I’d been awarded the 2013 Microsoft MVP Award for ASP.Net/IIS!  This is my sixth MVP Award and I’ve mentioned before I am extremely humble to be associated with a great group of less than 5000 technical people around the globe.  I’ll be aiming to continue my efforts to help the community and deliver as much content as I can to aid and spread knowledge. 

Categories: IIS | ASP.Net | MVP | Microsoft
Actions: E-mail | Permalink | Comments (2) | Comment RSSRSS comment feed

DDD North 2013 Announced and Session Submission Open

July 1, 2013 21:00 by Andrew Westgarth

On Saturday I attended the inaugural DDD East Anglia and took the ideal opportunity to announce the details for this year’s DDD North!  The event has been in the planning stages for a while and work continues apace.  I’m delighted to announce that we will be returning to the University of Sunderland on Saturday 12th October 2013 for DDD North.  The University of Sunderland’s Sir Tom Cowie’s Campus at St.Peters’ is a fantastic venue and was a fantastic place to study when I was studying for my degree many years ago.  We are working hard with the University of Sunderland, Sunderland Software City and Sunderland City Council to make sure this is the best DDD North yet! 

dddMedDDD North has launched a new site at http://www.dddnorth.co.uk and improvements and functionality is being added hourly at present as we ramp up our efforts.  The DDD North Call for Speakers is now open and will remain open till early-mid August before voting will commence so get your session submissions in now!  We’d love to see some new speakers submitting sessions as the original premise of DDD Events was to encourage and introduce new speakers.

DDD North is a free event and would not be so if it were not for the support of our fantastic sponsors.  If you know of any business or groups who would like to sponsor DDD North, we have a range of packages available and are open to discussions about suggestions, please get in touch using the contact form.

In the meantime keep an eye on www.dddnorth.co.uk for more news and sessions being added daily.  I look forward to seeing you at DDD North 2013 at the University of Sunderland!

Global Windows Azure Bootcamp April 27th 2013

March 11, 2013 21:05 by Andrew Westgarth

bootcamp

Are you interested in getting started with Windows Azure and have are free on Saturday April 27th 2013?  Then why not join me in Bradford at the Global Windows Azure Bootcamp being hosted by Black Marble?  I’ll be joining Microsoft Integration MVP, Robert Hogg, Windows Azure MVP, Steve Spencer and Microsoft Alliance Manager, Linda Hogg to deliver a one day deep dive class which will help you get up to speed with developing for Windows Azure.  This is a great opportunity to get real hands on experience with developing on Windows Azure from professionals who are actively developing solutions on the Windows Azure platform.  Windows Azure is a fantastic platform to develop on and we hope to see many of you at the event getting up to speed with development.

To register for the Bradford event please go to - http://www.blackmarble.co.uk/Register.aspx?Event=Global Windows Azure Boot Camp&Date=27-Apr-2013 and make sure to check out the list of pre-requisites for the bootcamp - http://globalwindowsazure.azurewebsites.net/?page_id=171 – please arrived prepared as we hate for you to miss out on the good stuff and spend the day getting your laptop ready Smile.

If you’re not able to get to Bradford but would still like to take part in the Global Windows Azure Bootcamp be sure to check out all the locations worldwide where other events are running -http://globalwindowsazure.azurewebsites.net/?page_id=151

Windows Azure IaaS Slides

January 18, 2013 19:46 by Andrew Westgarth

On Wednesday we celebrated the third birthday of NEBytes, a user group in the North East of England for Developers and IT Pros which I am a co-founder of.  We were delighted to partner up with the SharePoint User Group for the event and welcomed Dr. Bill Ayers to speak on Agile SharePoint Development.

In addition we’d planned to have a speaker visit to talk about Windows Azure IaaS at the event but unfortunately had problems sourcing a speaker due to a number of issues.  I therefore stepped in and reviewed some content from Microsoft and presented it at our event.  This was an interesting way to prepare for a presentation and I hope those who attended found the material and demos interesting and easy to follow.

I am currently spending a lot of time on the PaaS (Platform as a Service) side of Windows Azure and it was interesting for me to delve into the IaaS (Infrastructure as a Service) elements of Windows Azure.  There is quite a comprehensive offering and whilst I haven’t spent a huge amount of time in Amazon Web Services, apart from using an EC2 instance for Load Testing, I feel it’s growing fast and new features are coming on board frequently.

A few people asked for the slides after the presentation and here they are http://www.andrewwestgarth.co.uk/presentations/WindowsAzureVirtualMachines.pptx

Blog Now Hosted on IIS 8.0–DiscountASP.Net

September 22, 2012 11:04 by Andrew Westgarth

On Thursday night I was having an email conversation with Takeshi Eto from DiscountASP.Net about the hosting of my blog.  I’ve been hosting my blog with DiscountASP.Net for nearly five years and have been very, very happy with their service – always up to date often offering services faster than other hosters and very quick turn around of support tickets if ever I’ve had any issues – they also host the NEBytes site.

Well on Thursday I was asking about migrating my site onto IIS 8.0 hosting and enquiring when it would be available.  Well during the same conversation Takeshi posted a blog post announcing the arrival of Windows Server 2012 hosting to their US data centre with the team also working to bring it to their European datacentre -http://blog.discountasp.net/windows-2012-hosting-arrives-in-us-data-center/.  So the first thing I did was select the upgrade on my account and my site has now been migrated across to a new Windows Server 2012 platform and is running on IIS 8.0 Smile

Global ASP.NET Hosting Leader - Click Here

I also took the opportunity to upgrade the underlying platform of my blog, BlogEngine.Net, to 2.6.0.5 after not having upgrade from 2.0.  At the same time I’ve tinkered with the theme so I can present more content in my posts and I’ve also fixed a number of issues that I haven’t had time to investigate before now, for example my tweets are now displayed correctly in the bubbles above.  I’ve got a couple rough edges to smooth down like the search page but everything else seems ok.

DDD North 2012 Accommodation Suggestions

August 28, 2012 12:18 by Andrew Westgarth

As we are under an hour away from registration opening for DDD North 2012, which is being held at the Bradford University School of Management I thought it would be useful to put together a few links to help with accommodation choices.  This list isn’t extensive as last year’s event as it was held in my home town, but I will keep adding to this list based on comments and recommendations.

As I say I’ll keep adding this as I uncover more options.   For the agenda and more details about DDD North 2012 please visit http://bit.ly/dddnorth2

Categories: Events | DDD North
Actions: E-mail | Permalink | Comments (1) | Comment RSSRSS comment feed

Winners of the 2012 Imagine Cup UK Final are from the North East!

April 25, 2012 20:38 by Andrew Westgarth

Yes that’s right, today when I was at Microsoft UK Headquarters attending a day full of content on Visual Studio 11 and Team Foundation Server 11, the results of the 2012 Imagine Cup UK Finals were announced!  I heard the results were being streamed live at 4pm so when the time cam I hooked up to the broadcast and watched with bated breath for the results.  I’m delighted that both of the finalists from the Imagine Cup North East finals finished in the top two places, a fantastic achievement in the first year that the North East Colleges and Universities have been involved!

Teeside 0x32 came a very, very close second with their Cloud Doctor idea, a method of linking medical expertise around the world to provide help and assistance to those who don’t have access to medical treatment.  I remember seeing this idea presented impeccably at the Imagine Cup North East final a couple of weeks ago and I was blown away at how polished and ready the Web Application, Architecture and Windows Phone application was!  Well done to the guys for their fantastic idea, effort and hard work and I know that the judges had a very hard job in picking a winner because the ideas from both of the winners of the Imagine Cup North East finals were awesome!

The 2012 Imagine Cup UK Final winner is Team EyeWorks and they have created a combination of hardware and software to provide a Mobile Intelligent Retinal Analysis solution.  Combining Neural Nets, Windows Phones, Custom Lenses and Windows Azure they have come up with a fantastic idea to help with the early detection of diseases which cause loss of sight!  Again this is another fantastic idea and if you attended the NEBytes event in March you will have seen the team present their idea.  I honestly don’t know how the judges chose between the two because both ideas are fantastic!  Well done to everyone involved in Team EyeWorks and wish you all the very best at the 2012 Imagine Cup World Finals in Sydney later this year, do the North East and the UK Proud!!!



MCTS

Post calendar

<<  April 2014  >>
MoTuWeThFrSaSu
31123456
78910111213
14151617181920
21222324252627
2829301234
567891011

Disclaimer

The opinions expressed herein are my own personal opinions and do not represent my employer's view in anyway.

© Copyright 2014