So my colleague (same dude I did this and this with) showed up with another conundrum: we needed to update a feature service with data from a MySQL table and do so continuously with a 5 minute break in between. Now I don't like to think too much so naturally, I scoured the net for a solution that had already been done and I found this post by some good Esri folk where they updated a feature service through a Python script. This would have worked well for us - only that the feature service being updated was a Point feature service. The concept of updating a feature service works the same for all vector types but a Point service is markedly easier to update as all one needs to constitute a geometry type is X and Y coordinates. However, Line features and Polygon features have multiple points to take note of and thus are a bit harder to update.
I did not find anyone who had posted up a solution to the challenge of updating a polygon feature service using Python and therefore, once we had finished what we were doing for the client, we created this sample Python script to assist anyone who may be trying to do the same thing. The script is posted as a link at the bottom of the post. This post will primarily go through the code:
Step 1: Import the necessary libraries/modules for the organization
import urllib import urllib2 import json import sys import time import unicodedata
Step 2: Add credentials and feature service information. In this case, the feature service being updated is an ArcGIS Online hosted service.
# Credentials and feature service information username = "agol_username" password = "agol_password" service = "CBD_Parcels_Payment" # Note - the URL ends with "FeatureServer". The code below handles the /0 or /n for a given layer within the service fsURL = "http://services6.arcgis.com/HwQk19ysBarANM17/arcgis/rest/services/CBD_Parcels_Payment/FeatureServer" # Web Service URL to get updates from (IN JSON form) ISSURL = "http://cbddata.com/api/parcels" # How long to wait before updating again pauseTime = 300 # 300 seconds = 5minutes #variable for holding required object checkFeature = "" featureGeom = ""Step 3: Create a handler class for ArcGIS Online to generate tokens in order to connect to a Feature Service and query and update it
class AGOLHandler(object):
"""
ArcGIS Online handler class.
-Generates and keeps tokens
-template JSON feature objects for point
"""
def __init__(self, username, password, serviceName):
self.username = username
self.password = password
self.serviceName = serviceName
self.token, self.http, self.expires= self.getToken(username, password)
def getToken(self, username, password, exp=60): # expires in 60minutes
"""Generates a token."""
referer = "http://;www.arcgis.com/"
query_dict = {'username': username,
'password': password,
'referer': referer}
query_string = urllib.urlencode(query_dict)
url = "https://www.arcgis.com/sharing/rest/generateToken"
token = json.loads(urllib.urlopen(url + "?f=json", query_string).read())
if "token" not in token:
print(token['error'])
sys.exit(1)
else:
httpPrefix = "http://www.arcgis.com/sharing/rest"
if token['ssl'] is True:
httpPrefix = "https://www.arcgis.com/sharing/rest"
return token['token'], httpPrefix, token['expires']
Step 4: Create a function that allows the script to make a request for the hosted service - any type - update, add or delete procedure:
def send_AGOL_Request(URL, query_dict, returnType=False):
"""
Helper function which takes a URL and a dictionary and sends the request.
returnType values =
False : make sure the geometry was updated properly
"JSON" : simply return the raw response from the request, it will be parsed by the calling function
else (number) : a numeric value will be used to ensure that number of features exist in the response JSON
"""
query_string = urllib.urlencode(query_dict)
jsonResponse = urllib.urlopen(URL, urllib.urlencode(query_dict))
jsonOuput = json.loads(jsonResponse.read())
if returnType == "JSON":
return jsonOuput
if not returnType:
if "updateResults" in jsonOuput:
try:
for updateItem in jsonOuput['updateResults']:
if updateItem['success'] is True:
print("request submitted successfully")
except:
print("Error: {0}".format(jsonOuput))
return False
else: # Check that the proper number of features exist in a layer
if len(jsonOuput['features']) != returnType:
print("No features within the feature layer")
return False
return jsonOuput
Step 5: A function that queries the service layer and returns geometry and certain key attributes:
def fillEmptyGeo(con, fsURL, landRef):
"""
This function queries the service layer end points for a particular record in order
to return the.feature's geometry as well as the feature's object id
"""
ptURL = fsURL + "/0/query"
query_dict = {
"f": "json",
"where": "LRN='" + str(landRef) + "'",
"outFields": "*",
"token": con.token
}
print(query_dict)
checkFeature = send_AGOL_Request(ptURL, query_dict)
# Check if the queried value exists and give error if not
if (checkFeature) is False:
print("No such feature exists")
return ("", 9999)
else:
featureGeom = checkFeature["features"][0]["geometry"]
featureID = checkFeature["features"][0]["attributes"]["OBJECTID_1"]
print(featureID)
print("\n \n")
print 'Feature is present!!!'
print("\n \n")
return (featureGeom, featureID)
Step 6: Create a function for updating features once the record has been queried:
def updateFeature(con, ptURL, ptTime, featureItem, featuredID):
"""Use a URL, the connection token, the geometry of the feature, its object ID
value and its attributes to update an existing point."""
try:
print("Now updating")
print(featureItem["_Telephone"]+" ni nani")
print(featureItem)
# Updating Polygon
submitData = {
"features": [{
"geometry": ptTime,
"attributes": {
'OBJECTID_1': featuredID,
'Tenure': str(featureItem["_Tenure"]),
'Situation': str(featureItem["_Situation"]),
'ID': str(featureItem["_ID"]),
'Telephone': str(featureItem["_Telephone"]),
'LRN': str(featureItem["_LRN"]),
'Arrears': featureItem["_Arrears"],
'Value_': featureItem["_Value_"],
'MAP_SHEETN': str(featureItem["_MAP_SHEETN"]),
'Paid': str(featureItem["_Paid"]),
'PaymentsMade': featureItem["_PaymentsMade"]
}}],
"f": "json",
"token": con.token
}
print("\n \n")
print("Submitted data")
print(submitData)
print("\n \n")
jUpdate = send_AGOL_Request(ptURL, submitData)
except Exception as g:
print("ERROR caught: {0}".format(g))
return
Step 7: Create the main function that runs and calls all the prior functions. The procedure generally queries the database then loops through the results and updates each feature till the query results are ran through.
if __name__ == "__main__":
# Initialize the AGOLHandler for token and feature service JSON templates
con = AGOLHandler(username, password, service)
try:
# Check the Feature Service for the required feature and updates it according to the DB
# Loop indefinitely
while True:
# Get the current parcel values and read into memory
req = urllib2.Request(ISSURL)
response = urllib2.urlopen(req)
issPoint = json.loads(response.read())
for row in issPoint:
#get the geometry of the feature and its object ID field value
print(row["_LRN"])
retrievedGeom, featuredID = fillEmptyGeo(con, fsURL, row["_LRN"])
print(featuredID)
print(retrievedGeom)
# Update Polygon
updateFeature(con, fsURL + "/0/updateFeatures", json.dumps(retrievedGeom), row, featuredID)
if ((con.expires / 1000) - 61) < int(time.time()): # 60secs before token expires, get a new one
con.getToken(username, password)
#Check for reaching
print("Update process complete")
time.sleep(pauseTime)
# Generic exception handling: simple message is printed to the screen so the script continues to run.
# Additionally, an email or other action could be implemented below.
except Exception as e:
print("ERROR caught: {0}".format(e))
The comments on the code help to explain further what the code does. For further notes on working with Python, kindly check out this resource. The cumulative script for the updates can be found here. Feel free to download and modify for your own purposes. Happy coding!!
No comments:
Post a Comment