Access to ArcMap map document grid using Python and ArcObjects

When automating mapping production and exporting your map document layouts, you may need to manage map grids. They are grids or graticules which show the network of latitude and longitude lines in the layout.

For the very least, you would like to hide/show map grids in a map document’s data frame before exporting the map. The reason for this is that if your grid cell is of 1×1 km size, when exporting the map layout being in a small extent (for instance, 1: 1,000,000 you will get a pretty much no map except a black image because the grid will take all the place.

Being able to access map grids can also be handy when you have multiple grids with different grid cell sizes and you want to be able to control at what map scales each of the map grids should be visible.

Unfortunately, you cannot use arcpy to access grids defined for a data frame. However, the grids are exposed via an ArcGIS extension called Production Mapping which according to the Esri Help page

streamlines your GIS data and map production by providing tools that facilitate data creation, maintenance, and validation, as well as tools for producing high-quality cartographic products.

You might be surprised to learn this, but Production Mapping extension has its own arcpy-like site-package called arcpyproduction. Within this package, there is a class Grid class which provides access to grid properties for a grids and graticules layer. However, if you don’t have Production Mapping extension (it might be hard to justify buying this extension if you are not involved in heavy map production), as map grids are not exposed via arcpy, you have to use ArcObjects.

Since many would like to automate showing/hiding map grids when exporting map layouts, I’ve written a tiny function which can do that using Python. I have blogged earlier about how to start using ArcObjects from Python, so make sure to read this post first. Using this code, you will be able to choose which map grid you would like to use (if you have multiple grids with different cell size defined) when exporting a map layout.

Python progression path for GIS professionals

Over last years, I was working with Python almost full time either scripting some desktop GIS workflows or developing code for the back-end geoprocessing services using arcpy. I learned all kinds of Python packages, everything from data science packages such as pandas and numpy to more widely applicable ones such as xlsxwriter and reportlab. Being able to find a package and start using it producing the outputs needed in a matter of minutes is one of the key selling points of Python, I think.

However, due to the presence of such a large number of resources that are related to Python (just check this repository on GitHub – A curated list of awesome Python frameworks, libraries, software and resources)- one might feel a bit lost. There are so many things to learn, which are the most important ones? It also makes things a bit more complicated for niche developers or GIS analysts who do Python programming just occasionally. I have also experienced frustration being unable to identify the key competence areas to focus on and how to track my progress. Am I learning Python packages that are relevant for geospatial operations? What else should I learn after I’ve managed a certain feature of the language or a framework?

The result of this thought process is a public repository on GitHub which I am working on. It’s called Progression path for a GIS analyst who wants to become proficient in using Python for GIS: from apprentice to guru which is inspired partially by the awesome-python and partially by a SO post Python progression path – From apprentice to guru.

This is an attempt to provide a structured collection of resources that could help a GIS professional to learn how to use Python when working with spatial data management, mapping, and analysis. The resources are organized by progress category so basically everyone should be able to learn something new along the way. The resources will include books, web pages and blog posts, online courses, videos, Q/A from GIS.SE, links to code snippets, and some bedtime readings.

Be sure to check this one out, pick a topic of interest and start working on it. Also, feel free to star the repository if you have a GitHub account 🙂

4 years with GIS at StackExchange

It’s been 4 years since I’ve joined the GIS.SE community – one of the sites under the StackExchange umbrella. It’s a good time to reflect a bit I guess. I have learned a lot during those years reading answers provided by peer GIS professionals. I have also learned quite a lot while answering questions for which I thought I could provide a working solution. I have never thought I would get so many reputation points though. Yet I have now just over 20K (which places me in the top 20 users of the GIS.SE site), which is quite amazing as I have never thought that my answers could get so many up votes.

In one of the episodes of one of my favorite Python podcasts – Talk Python To Me – Martijn Pieters provided his thoughts on why he enjoys answering questions on StackOverflow. I’ve decided to share my arguments why being involved in GIS.SE is so great:

  • Answering questions is a great exercise which you can do just for fun. For instance, someone has asked a question – Displaying attributes upon their polygons within ArcMap legend?. I have never done it myself, so I got curious whether this could be done. After searching around in the UI for some time, I finally found the sequence of steps to perform and shared it with the questioner. Great.

 

  • An answer can be a form of organizing own knowledge about a subject. For instance, someone has asked a question – Formatting street network data for ArcGIS Network Analyst?. I have been working with Network Analyst for many years but never have compiled a short “getting started guide” on where to find data sources, what skills are necessary etc. Answering this question gave me a chance to put all my reference information into one place which other users can also find and read.

 

  • An answer can help someone who has faced a very peculiar problem to get a solution quickly without spending hours if not days. For example, someone has asked *.lyr to *.style conversion in ArcGIS. I needed to do exactly this just a few weeks earlier and was searching for the tool to do this for some days. Being able to answer a question in 10 minutes after it was posted knowing that someone else wouldn’t need to go through all possible UI tools in ArcMap is incomparable.

 

  • An answer I give to others can be an answer I will be looking for in the future myself. It is funny, but sometimes, when I have a question I am trying to find an answer to, when using a search engine, I find the GIS.SE page with the answer I just needed and a sample code snippet I can use. I am so pleased someone has found the time to post it. Looking at the author of the answer makes me laugh because it was me.

 

  • Answering questions makes me a better writer as I need to formulate my thoughts very accurately and convey the information in a clear and consistent way. I should also be concise yet comprehensive. I believe answering on GIS.SE helped me to become a better writer which I think is a great skill to have.

 

  • Answering questions makes me a more competent professional as I learn a lot about the GIS software and GIS programming. I have answered many arcpy related questions while I was learning arcpy myself. In addition, some questions gave me a chance to play with a software I don’t use daily which helps me to stay up-to-date on what software is being used by other people and how.

    To summarize, over last 4 years, I learned a lot about GIS software features, helped many people with their tasks, improved my writing skills, and became a more competent GIS expert. Not too bad!

    PS. Go ahead and read a great meta post How do high reputation (>5,000) users find time to commit to GIS Stack Exchange?

Update field aliases in map document layers

If you have been using arcpy for a while, you might have found that there are certain limitations. For instance, you cannot:

  • create a new map document;
  • create/modify map document layout elements (for instance, change scale bar units or labels);
  • create map grid /graticules in layout;
  • edit network dataset properties;

Another fairly basic thing that you cannot do is to view/edit field aliases for layers in a map document. This is a quite common scenario when you have authored a map document using a feature class from a geodatabase. By default, the field aliases for the map layers you create are generated using field aliases of the feature class fields stored within the geodatabase. However, as soon as you’ve created a map layer from a feature class, the layer fields have no connection to their data source fields aliases.

This means that if you would change the alias of a feature class field, this won’t be reflected in the layer properties that refer to the feature class. Of course, you can always manually update the field aliases using Layer Properties window in ArcMap, however this can get complicated if you are not sure what fields should be updated.

This is because it is currently (ArcGIS 10.4.1) impossible to list the field aliases for a map layer. When you are using arcpy.ListFields() function, you won’t be able to get aliases for the fields that you can see from the Layer Properties window. This is because this function will pull the field aliases from the layer’s data source, that is a feature class.

This implies that if you have a bunch of map documents and you would want to update the map layers field aliases for them in bulk, you cannot use arcpy. You have two options really:

  1. Use X-Ray for ArcMap 10.1 add-in. Then you can open every map document file you want to apply changes to and use the command on the add-in’s toolbar.
  2. Use ArcObjects to access the map document file programmatically, make required changes in the layer field properties and save it.

I think it would be great if we could make those changes using arcpy. A common use case is when you have a suite of ArcGIS Server map services you maintain and in a case when a geodatabase feature class field alias is changed, you would like to reflect it in the service as well. So, you would need to update the field aliases for the map documents and then republish them. Obviously, you would like to script this.

Unless you would like to build a custom tool in .NET/Java, you would call comtypes package from Python to invoke ArcObjects COM methods. I have blogged about using comtypes for accessing ArcObjects from Python before, so be sure to check this post.

Here is a snippet of code I use for automating field aliases updates. I hope this can save you some time.

 

 

Load Esri geodatabase tables into SQLite

I have blogged earlier about loading Esri file geodatabase feature classes into SQLite database by using standard geoprocessing tools. This time, I’ve written a short sample that creates a SQLite database, reads a file geodatabase feature class rows, and then creates a new table and insert the features into it. Everything except reading file geodatabase (I use arcpy to access features with a cursor) is done using pure Python and SQL. The logic is rather simple – you read the schema of a geodatabase table and then mirror it inside a SQLite database table. Then you insert the features’ geometry and attributes into right columns.

I was pleased to observe excellent performance of data insertion and query processing. Of course, the initial data load is an overhead that will present in the script, but the efficiency of data manipulation using SQL and spatial analysis using ST_Geometry functions outweighs it, in my opinion. If you will work with rather larger datasets, you should consider tuning the performance of SQLite spatial indices because using ST_Geometry may be initially slow.

It is also worth noting that with some minor modifications, it would be possible to load a non-spatial file geodatabase table into an SQLite database table if you want to perform some SQL-based data aggregation. A very unique feature of SQLite database is that it is possible to generate such a database completely in memory without writing anything on disk. This is suitable for scripts where you want to do the processing on the fly without creating any additional scratch workspaces. Use the syntax for creating SQLite databases in memory (:memory:). I’ve done tests loading the spatial extension and some data and it worked fine.