Sunday, 24 July 2016

Flashing a graphic on an ArcGIS API for JavaScript map

Hi people! It's been a while since I last posted, but I am definitely not falling off - just been a bit busy. Still not an excuse for not posting. Now that the apology is out of the way, let's get back to these posts, shall we?

Recently, a client requested us to make an application where he would be able to query points and zoom to them individually. OK, no problem. However, he had an additional requirement, he wanted the queried graphic to 'flash' after being zoomed to because it would be in the midst of other points and would need to be distinguishable from the rest. Though we asked the client to consider highlighting the point, he would have none of it and insisted on the flashing requirement. As my manager always says - the customer is always right, even if he is wrong. We therefore got to work on it and the result, however unflattering it may be, is what we will look at today.

Before we start, just in case someone is wondering what I mean by flashing , the GIF below should be of some assistance. The GIF is an example of a flashing graphic, derived from an implementation of the same from the browser based version of Operations Dashboard for ArcGIS.

Yes, that's what I mean by flashing

SO how do we go about flashing a graphic? A while back, this method would have helped us get there quick but sadly, the times they-are-a-changing. That said, the concept remains the same and this is what we shall focus on understanding. 

The method that we ended up using relies heavily on dojo.Animation - the foundation class for all Dojo animations. It provides several simple methods good for controlling your animation, such as play, pause, and stop. Several types of animation are made possible through this class but our main interest was in two of these methods - namely the fadeIn and the fadeOut methods. To use these functions, we included the dojo.fx module:


The fadeIn and fadeOut methods do exactly what you think they do - they make the graphic appear to be fading in and fading out. These are exactly the type of effects that will be used to make our graphic appear to be flashing. In the example in the jsfiddle below, the map loads a FeatureLayer on to it and at the click of the button on its top right hand corner, a random graphic flashes.




The following is the procedure that was followed:

After loading the map, a point FeatureLayer is added to it - no fancy properties need to be placed in the constructor. The bulk of the work, if not all of it, actually happens when one clicks the 'Flash a point' button.

Clicking the button begins the flashing process. In this application, only one point is flashed at a time therefore the graphic must be identified. Since we intend to flash random points in the layer, we first determine how many points there are in the layer using the following line:


Once we know how many points the layer contains, we can now apply a formula to make sure each time the flash button is clicked, a random feature within the layer is picked and flashed.


Once our random graphic has been identified, we can now get to the bit of flashing it. (Please note that in a more realistic case, the graphic to be flashed will be determined through a query that fits certain parameters. This is just an example). The next thing we do is add this graphic to the default graphics layer(map.graphics) or a graphics layer that you have designated.

The animateGraphic method/function takes in the graphic and applies the dojo methods that we talked about earlier. Both fadeIn and fadeOut methods have a standard set of properties for specifying animation settings, some of these include:

  • node - The domNode reference or string id of a node to apply the animation effects to. In our case, we use the .getDojoShape().getNode() graphics method to get the node by which our graphic is referenced by.
  • duration - how long the animation will run in milliseconds. In this case we chose to make both fadeIn and fadeOut animations last 0.7 seconds.
  • easing - A timing function applied to the animation effect. In our example, the fading is applied linearly.
As seen below, these properties are applied to both functions.



All that code only accounts for one fadeIn and fadeOut cycle. Hoowever, we'd like to have three of those in quick succession. What to do? Probably what Aretha Franklin talked about in the hook of this 1968 hit. Enter the dojo.fx module and the chain convenience function which merges the animations into a single animation, playing back each of the animations one right after the other.
In our example, we create an array of animations based on repeating animations A and B three times, chain that together and then play the animation - resulting in a graphic that flashes 3 times then disappears.


(Though our example ends there, it may be prudent to clear graphics once you are done flashing them).

Well then, I hope you have enjoyed getting to know how we implemented flashing of graphics in our application. Feel free to point out any ways our code can be made cleaner or faster.

Happy coding!!

Tuesday, 12 July 2016

Updating a feature service from an external source using Python

Hi guys! It's been more than a week since I last posted and I am sorry for that - though no one noticed (scowl). Anyways, your boy is back and has a new solution to post up. Only this time it's a python solution!

So my colleague (same dude I did this and this with) showed up with another conundrum: we needed to update a feature service with data from a MySQL table and do so continuously with a 5 minute break in between. Now I don't like to think too much so naturally, I scoured the net for a solution that had already been done and I found this post by some good Esri folk where they updated a feature service through a Python script. This would have worked well for us - only that the feature service being updated was a Point feature service. The concept of updating a feature service works the same for all vector types but a Point service is markedly easier to update as all one needs to constitute a geometry type is X and Y coordinates. However, Line features and Polygon features have multiple points to take note of and thus are a bit harder to update.

I did not find anyone who had posted up a solution to the challenge of updating a polygon feature service using Python and therefore, once we had finished what we were doing for the client, we created this sample Python script to assist anyone who may be trying to do the same thing. The script is posted as a link at the bottom of the post. This post will primarily go through the code:

Step 1: Import the necessary libraries/modules for the organization

import urllib
import urllib2
import json
import sys
import time
import unicodedata

Step 2: Add credentials and feature service information. In this case, the feature service being updated is an ArcGIS Online hosted service.

# Credentials and feature service information
username = "agol_username"
password = "agol_password"    
service = "CBD_Parcels_Payment" 

# Note - the URL ends with "FeatureServer". The code below handles the /0 or /n for a given layer within the service
fsURL = "http://services6.arcgis.com/HwQk19ysBarANM17/arcgis/rest/services/CBD_Parcels_Payment/FeatureServer"  

# Web Service URL to get updates from (IN JSON form)
ISSURL = "http://cbddata.com/api/parcels"

# How long to wait before updating again
pauseTime = 300  # 300 seconds = 5minutes

#variable for holding required object
checkFeature = ""
featureGeom = ""

Step 3: Create a handler class for ArcGIS Online to generate tokens in order to connect to a Feature Service and query and update it

class AGOLHandler(object):    
    """
    ArcGIS Online handler class.
      -Generates and keeps tokens
      -template JSON feature objects for point
    """
    
    def __init__(self, username, password, serviceName):
        self.username = username
        self.password = password
        self.serviceName = serviceName
        self.token, self.http, self.expires= self.getToken(username, password)  

    def getToken(self, username, password, exp=60):  # expires in 60minutes
        """Generates a token."""
        referer = "http://;www.arcgis.com/"
        query_dict = {'username': username,
                      'password': password,
                      'referer': referer}

        query_string = urllib.urlencode(query_dict)
        url = "https://www.arcgis.com/sharing/rest/generateToken"
        token = json.loads(urllib.urlopen(url + "?f=json", query_string).read())

        if "token" not in token:
            print(token['error'])
            sys.exit(1)
        else:
            httpPrefix = "http://www.arcgis.com/sharing/rest"
            if token['ssl'] is True:
                httpPrefix = "https://www.arcgis.com/sharing/rest"
            return token['token'], httpPrefix, token['expires'] 

Step 4: Create a function that allows the script to make a request for the hosted service - any type - update, add or delete procedure:

def send_AGOL_Request(URL, query_dict, returnType=False):
    """
    Helper function which takes a URL and a dictionary and sends the request.
    returnType values = 
         False : make sure the geometry was updated properly
         "JSON" : simply return the raw response from the request, it will be parsed by the calling function
         else (number) : a numeric value will be used to ensure that number of features exist in the response JSON
    """
    
    query_string = urllib.urlencode(query_dict)

    jsonResponse = urllib.urlopen(URL, urllib.urlencode(query_dict))
    jsonOuput = json.loads(jsonResponse.read())

    
    if returnType == "JSON":
        return jsonOuput
    
    if not returnType:
        if "updateResults" in jsonOuput:
            try:            
                for updateItem in jsonOuput['updateResults']:                    
                    if updateItem['success'] is True:
                        print("request submitted successfully")
            except:
                print("Error: {0}".format(jsonOuput))
                return False
            
    else:  # Check that the proper number of features exist in a layer
        if len(jsonOuput['features']) != returnType:
            print("No features within the feature layer")
            return False
            
    return jsonOuput

Step 5: A function that queries the service layer and returns geometry and certain key attributes:

def fillEmptyGeo(con, fsURL, landRef):
    """
    This function queries the service layer end points for a particular record in order
    to return the.feature's geometry as well as the feature's object id 
    """
        
    ptURL = fsURL + "/0/query"
    
    query_dict = {
        "f": "json",
        "where": "LRN='" + str(landRef) + "'",
        "outFields": "*",
        "token": con.token
    }

    print(query_dict)

    checkFeature = send_AGOL_Request(ptURL, query_dict)
    # Check if the queried value exists and give error if not
    if (checkFeature) is False:
        print("No such feature exists")
        return ("", 9999)
    else:
        featureGeom = checkFeature["features"][0]["geometry"]
        featureID = checkFeature["features"][0]["attributes"]["OBJECTID_1"]
        print(featureID)
        print("\n \n")
        print 'Feature is present!!!'
        print("\n \n")
            
    return (featureGeom, featureID)

Step 6: Create a function for updating features once the record has been queried:


def updateFeature(con, ptURL, ptTime, featureItem, featuredID):
    """Use a URL, the connection token, the geometry of the feature, its object ID
    value and its attributes to update an existing point."""
 
    try:
        print("Now updating")

        print(featureItem["_Telephone"]+" ni nani")
        
        print(featureItem)
        
        # Updating Polygon
        submitData = {
            "features": [{
            "geometry": ptTime,
            "attributes": {
                'OBJECTID_1': featuredID,
                'Tenure': str(featureItem["_Tenure"]),
                'Situation': str(featureItem["_Situation"]),
                'ID': str(featureItem["_ID"]),
                'Telephone': str(featureItem["_Telephone"]),
                'LRN': str(featureItem["_LRN"]),
                'Arrears': featureItem["_Arrears"],
                'Value_': featureItem["_Value_"],
                'MAP_SHEETN': str(featureItem["_MAP_SHEETN"]),
                'Paid': str(featureItem["_Paid"]),
                'PaymentsMade': featureItem["_PaymentsMade"]
                }}],
            "f": "json",
            "token": con.token
        }
        print("\n \n")
        print("Submitted data")
        print(submitData)
        print("\n \n")
        
        jUpdate = send_AGOL_Request(ptURL, submitData)          
  
    except Exception as g:
        print("ERROR caught:  {0}".format(g))

    return



Step 7: Create the main function that runs and calls all the prior functions. The procedure generally queries the database then loops through the results and updates each feature till the query results are ran through.

if __name__ == "__main__": 

    # Initialize the AGOLHandler for token and feature service JSON templates
    con = AGOLHandler(username, password, service)
    
    try:
        
        # Check the Feature Service for the required feature and updates it according to the DB
        
        # Loop indefinitely  
        while True:
            
            # Get the current parcel values and read into memory
            req = urllib2.Request(ISSURL)
            response = urllib2.urlopen(req)
            
            issPoint = json.loads(response.read())

            for row in issPoint:
                #get the geometry of the feature and its object ID field value
                print(row["_LRN"])
                retrievedGeom, featuredID = fillEmptyGeo(con, fsURL, row["_LRN"])
                print(featuredID)
                print(retrievedGeom)
                # Update Polygon
                updateFeature(con, fsURL + "/0/updateFeatures", json.dumps(retrievedGeom), row, featuredID)
                
                
            if ((con.expires / 1000) - 61) < int(time.time()):  # 60secs before token expires, get a new one
                con.getToken(username, password)
    
            #Check for reaching
            print("Update process complete")
           
            time.sleep(pauseTime) 
                
            
    # Generic exception handling: simple message is printed to the screen so the script continues to run.
    # Additionally, an email or other action could be implemented below.
    except Exception as e:
        print("ERROR caught:  {0}".format(e))



The comments on the code help to explain further what the code does. For further notes on working with Python, kindly check out this resource. The cumulative script for the updates can be found here. Feel free to download and modify for your own purposes. Happy coding!!