Today, I would like to focus on using the arcpy.mapping module. This module is included in the ArcPy package and its primary use is to manage map documents (.mxd) and layer files (.lyr). This module can be used very effectively when data sources in mxd map documents need to be updated or redirected to some other locations.

To learn more about the arcpy.mapping module, please refer to the Esri help page: Updating and fixing data sources with arcpy.mapping

Of course, you can change the source paths using ArcCatalog, yet it does not provide the same level of access to underlying objects as the ArcPy package. Please refer to the Esri Knowledge Base article and Help pages to learn more how to change data sources in your .mxd map documents:

HowTo: Use the ‘Set Data Source’ tool for map documents in ArcCatalog and Setting data sources respectively.

Below is the script which can be used for changing multiple SDE data sources in one .mxd map document.

import arcpy
# Define the path to the .mxd map document you want to update
mxd = arcpy.mapping.MapDocument(r"C:\Temp\Esri_sde_service.mxd")

# Define which connection strings will be changed. Here there is just one connection string being changed.
mxd.findAndReplaceWorkspacePaths(r"C:\Users\username\AppData\Roaming\ESRI\Desktop10.0\ArcCatalog\Production@sde.sde",
r"C:\Users\alte\AppData\Roaming\ESRI\Desktop10.0\ArcCatalog\Development@sde.sde")

# It is possible to add more connection strings that have to be changed in the same map document
mxd.findAndReplaceWorkspacePaths(r"C:\Users\username\AppData\Roaming\ESRI\Desktop10.0\ArcCatalog\RasterData.sde",
r"C:\Users\username\AppData\Roaming\ESRI\Desktop10.0\ArcCatalog\RasterData_load.sde")

# The path where the new .mxd map document will be saved to.
mxd.saveACopy(r"C:\GIS\Output\Sde_load.mxd")
del mxd

Here is the script which can be used for changing multiple SDE data sources in multiple.mxd map documents.

import arcpy, os
# Define the path to the folder where multiple .mxd map documents are stored
folderPath = r"C:\GIS\Maps"
for filename in os.listdir(folderPath):
fullpath = os.path.join(folderPath, filename)
if os.path.isfile(fullpath):
basename, extension = os.path.splitext(fullpath)
if extension.lower() == ".mxd":
mxd = arcpy.mapping.MapDocument(fullpath)

# Define which connection strings will be changed. Here there are two connection strings being changed. One for vector data
mxd.findAndReplaceWorkspacePaths(r"C:\Users\username\AppData\Roaming\ESRI\Desktop10.0\ArcCatalog\Production@sde.sde", r"C:\Users\username\AppData\Roaming\ESRI\Desktop10.0\ArcCatalog\Development@sde.sde")

# and second one for raster data which may have used another connection string.
mxd.findAndReplaceWorkspacePaths(r"C:\Documents and Settings\username\Application Data\ESRI\ArcCatalog\raster_data.sde", r"C:\Documents and Settings\username\Application Data\ESRI\Desktop10.0\ArcCatalog\raster_dataV10.sde")

# You can use even more defitions of the connection strings that will be changed. Note that here is the update of the path from 9.3 to 10 is going on which you can see since the path to the .sde file is different (update from 9.3 to 10)
mxd.findAndReplaceWorkspacePaths(r"C:\Documents and Settings\username\Application Data\ESRI\ArcCatalog\TopologyData.sde", r"C:\Documents and Settings\username\Application Data\ESRI\Desktop10.0\ArcCatalog\TopologyData_manager.sde")
mxd.save()
del mxd

Let’s start with what benefits JavaScript has:

• it seems to show the highest application performance compared to other APIs;
• it supports the biggest number of operating systems;
• since this API uses pure HTML and JavaScript there are no browser plug-ins required;
• JavaScript developers have access to multiple JavaScript libraries and frameworks like Dojo framework which includes various charting tools and common controls;
• there are some free web-based development environments like Aptana Studio and any text editor can be used as well;
• Esri provides access to multiple out-of-the-box ArcGIS.com viewers.

For those who have just decided to learn JavaScript, visit the NetTuts+ web site, which has a great article on how to master JavaScript. Personally, I had a great time working through the courses at codeacademy.com – a terrific Web site where you can learn to programme, whilst having fun at the same time.

You can get started with the ArcGIS API for JavaScript at their home page. JavaScript API is hosted by ESRI on ArcGIS Online and is naturally available for free use. However, you can also download and install the developer kit which includes the API reference, samples, and documentation.

NB: The SDK and the API reference of previous versions are NOT available online yet – it can be downloaded and used locally at the ArcGIS API for JavaScript Download Page.

I have also thought to share some resources which are worth a look. I have organized them in a list with a concise description to give you a general understanding of the purpose.

• Aptana Studio is a good free web development framework. You can get “Intellisense” in the Aptana Studio similar to what you get when working in the Visual Studio and using ArcGIS API for Silverlight. Here are the instructions how to integrate the Dojo and ArcGIS API for JavaScript syntaxes into Aptana Studio as well as how to enable JSLint code validation.

• Dojo is a very large JavaScript library with lots of components that one can integrate into your own applications.

• A great collection of best practices in web development is available on the Mozilla Web site.

• Jsonlint is a very popular Json validator.

• JS Beautifier is a neat tool that organizes JavaScript/HTML code in a nice layout.

• JSLint is another good resource for code quality validation and finding typos.

• Google Chrome Developer Tools is a great web development toolset from Google.

• JQuery is a very good JavaScript library widely used with the ArcGIS API for JavaScript.

• Esri ArcGIS Server Blog has many posts regarding the JavaScript API. Here you can find information on how JavaScript API can be used together with various freely available online viewers.

And just one more thing for those who need to check the code coming in multiple files. WinDiff is a very neat, free utility for comparing the contents of the HTML/JS files to figure out in what parts they differ.

Happy coding!

ScrewTurn Wiki makes it possible to create, manage, and share documentation, files, and other assets via the wiki concept. A wiki is a collaboratively-edited, information-centered website: the most famous one is Wikipedia, as stated by the ScrewTurn Wiki homepage. The ScrewTurn Wiki is hosted on the IIS Web server and can be accessed via a standard Web browser (for instance, IE or Firefox). The ScrewTurn Wiki per se is a folder-based Web application deployed on the IIS Web server.

ScrewTurn Wiki is based on Microsoft ASP.NET 3.5 technology, meaning that you need a Windows operating system to run it, such as Windows XP, Windows Server 2003 or 2008.

The following software must be installed on the server in order to run ScrewTurn Wiki:

• Internet Information Services (IIS) 5.1/6/7/7.5 (is free, since it is included in the Windows; can be installed via Adding/Removing Programs menu in Windows)
• Microsoft .NET Framework 3.5 SP1 (ASP.NET 3.5 SP1) (is free; can be downloaded from the Microsoft Web site)

ScrewTurn Wiki can be extended and has the GUI which is very intuitive, yet the functionality is rich. Source code of ScrewTurn Wiki is available as Visual Studio 2008 solution, MSBuild script, and reference documentation (CHM), which implies that any required customization can be done in the source code and then deployed to the production. Customization of the interface can be done via an Administrative web interface or by using stand-alone plug-ins.

I have been using it for a while now and can highly recommend the ScrewTurn wiki for anyone who is in need of such a compact, neat, and customizable web-based system for organizing and sharing information. Feel free to go ahead and download the wiki or buy the commercial license, thus supporting the project.

Happy Wiking!

To deploy a Silverlight application with RIA services on Windows 7/Windows 2008, follow the steps outlined below:

1. Run Visual Studio as Administrator: Right-click the Visual Studio shortcut and choose Run as Administrator. This is required for obtaining proper system permissions when publishing applications to the IIS inetpub folder. If you want to set up always running the Visual Studio 2010 as administrator in Windows 7, go to the Microsoft Visual Studio 2010 shortcut, right-click and choose Properties. Then switch to the Compatibility tab and check the Run this program as an administrator option in the Privilege level section.

2. Open the Silverlight solution: File > Open Project/Solution.

3. Right-click the ProjectName.Web and choose Publish. Choose as publish method the File System. Point to the folder to which the application will be published. Check the Delete all existing files prior to publish to make sure that all existing files within the folder will be deleted.

4. Open IIS Manager: To open IIS Manager from the Start menu – click Start, and then click All Programs. Click Administrative Tools, and then click Internet Information Services (IIS) Manager. To open IIS Manager from the Search box – click Start. In the Start Search box, type inetmgr and press ENTER. Go to the Default Web site. Right-click the folder where the application was published to and choose Convert to application. In the Add application dialog box, choose ASP.NET v4.0 application pool and not ASP.NET v4.0 Classic.

If for some reason, you are missing these pools even though you have .NET Framework 4 installed, you can fix this by registering ASP. Open your command prompt and run: cd C:\Windows\Microsoft.NET\Framework\v4.0.30319\. Then run: aspnet_regiis.exe -ir.

5. Right-click the application folder in the IIS manager and select Switch to content view.

Select .html or .aspx file and choose Browse in the Actions panel.

6. You will get an error message, because the RIA service was not created yet on-the-fly.

7. In order to make the application run as designed, you have to create a service in the Web browser first. Open an instance of Internet Explorer, and go to the URL:

http://machinename/appname/<Projectname>-Web-Services-<name of the service file>

Name of the service file can be obtained from the Visual Studio Solution Explorer: Go to <ProjectName>.Web and then expand Services folder to see the name of the created service.

Your URL may look like this: http://localhost/SL_Work/SlEventManager-Web-Services-EventManagerDomainService.svc. The URL may look different depending on the structure of the files in the Solution Explorer, so adopt this example. Going to this URL is crucial because you will not be able to view the application correctly without creating the service first.

Select .html or .aspx file in the IIS Manager and choose Browse in the Actions panel.

8. If you get an error saying related to the local user profile, check what authentication settings you have for the application. Opet IIS Manager and go to your application and then select Authentication in the Features view. You should have disabled: ASP.NET Impersonation, Windows Authentication and Basic Authentication. You should have enabled: Forms Authentication and Anonymous Authentication.

9. Now you can select .html or .aspx file in the IIS Manager and choose Browse in the Actions panel.

Hopefully it works!

The course is basically about how to manage the GIS data in a SQL Server database efficiently, as well as how to configure and optimize the ArcSDE for Microsoft SQL Server in order to achieve the best possible performance. I found the course to be concise, yet very informative. It gives a very good understanding of what ArcSDE and Microsoft SQL Server are, what functionality each of them provides, and how they are integrated in the ArcGIS Server system. I think one would learn quite a lot should one go to this course, coupled with the Data Management in the Multiuser Geodatabase course, because the latter course gives an idea about certain topics, which are not covered in the exercises of the former – such as: archiving, replication, setting permissions to the datasets, and managing versions.

Here are some additional reflections that I have concerning the course material, which I think might be valuable to others:

1. Quite an important topic when working with ArcSDE is how the spatial database definitions (which are available in the Catalog tree under the Spatial Database Connections folder in the ArcCatalog) are stored on the disk. These files are stored in the system user folder and can be copied or moved to another place and thereafter used for adding data or setting up the source paths to other applications like ArcMap.

2. Even though, in ArcSDE 10.1 release, Esri is supposed to provide graphical interface for managing ArcSDE instances, which will be a great time-saver, I still think that it is worth familiarizing yourself with the most commonly used ArcSDE commands.

3. Another issue that might arise after the data have been registered as versioned is how to unregister the data to be versioned. There is no such option to do this directly in the context menu, so one has to go to the Customize menu and add the Unregister as versioned button (under Geodatabase Tools category) to the user interface and use it when needed.

4. To be aware of spatial reference management in the geodatabase is very important too. A feature dataset can contain feature classes in only one coordinate system, so it is impossible to store feature classes with different a coordinate system in the same feature dataset. When importing a feature class from another geodatabase or a stand-alone shapefile to a feature dataset, it will be projected on-the-fly to match the feature dataset’s spatial reference system.

I got a chance to work with some pilot project data which consists of a several thousand points representing geographic locations. The functionality of the software allows you to solve different routing problems including, but not limited to, Vehicle Routing Problem (VRP), very efficiently and provides tremendous opportunities for tuning and advanced routing customization. I was personally impressed with its performance in generating routes between the stops – I was literally blown away by the whole thing.

Those of you who work in the UK with newspaper distribution, waste collection, snow plowing or other public works, where you have multiple vehicles and large human resources that need to be effectively managed, take a second to read a bit about RouteSmart for ArcGIS at the RouteSmart UK Web site or contact James Holt who is the Technical Director at the Integrated Skills Ltd, the authorized distributor of the RouteSmart in the United Kingdom. I personally got the chance to meet up with James last week to take a look at RouteSmart and I can recommend him very highly.

Originally, I thought of preparing a classic Microsoft PowerPoint presentation. I should acknowledge that this is a well-established standard in academic sphere as well as in the business world. PowerPoint is great for assembling pieces of information into a sequential group of slides, yet it is not very dynamic and flexible because users are not provided with the functionality for quick jumping between slides and options for interactions with the slides are limited.

I have read a bit about Prezi before, a web-based tool for creating dynamic Flash-alike presentations. So, I grabbed a book in the library and spent a couple of evenings digging into the new environment for presenting information. I was genuinely impressed and can highly recommend this software. The book I used – Prezi for Dummies (even though I am a bit skeptic towards this “dummy” series) – turned out to be great. The book contains plenty of useful technical tips on creating presentations as well as some concepts on how to create a good presentation (here I can recommend another great book – The craft of scientific presentations) , audience-speaker interaction, and human cognition.

Prezi has different licenses including one you can use for free; the only trade-off here is that the presentation will get Prezi logo picture. You may create a presentation online and then present your work either from the Internet or offline by downloading the presentation files. Keep in mind though that it is based on Flash, so be sure to install the required plug-ins and/or Adobe Flash player.

First of all, to get started working with street data is not as easy as it seems. One may need to contact a local agency for any street data in the public domain. At the same time, quite often there are no street data available, which implies spending quite a large amount of money on data acquisition. Technically, to get started performing network analysis using ESRI Network Analyst extension is relatively easy, yet to get the data for the work is much harder.

Secondly, there is a lack of awareness of street data sources available in the ESRI community. Obviously, there are much fewer people working with Network Analyst extension compared to the users of Spatial Analyst or 3D Analyst extensions. So, a person involved in solving routing tasks is forced to get to know the software on his/her own by using limited help system and other resources available. The number of white papers and magazine articles related to the ESRI tools for network analysis is limited as well, yet there is a great deal of interest in this particular area.

Finally, vendors’ street data is shipped in a raw format and obviously not in ArcGIS network dataset format! Navteq Navstreets and TeleAtlas Multinet products are delivered among others in ESRI Shape format. This implies that users may need to dive into the hundreds of pages which comprise the reference manual, in order to figure out many things – such as how to convert turns tables into ArcGIS turn feature class, import signposts, and set the directions parameters properly, to name a few. It is a time consuming process which is likely to involve contacting vendor’s support and extensive trial by error tests.

To give a thorough overview of the network analysis in ArcGIS would take too much time, so let’s just focus on the acquisition of street data in this post. Options available are as below/follows:

1) TIGER (Topologically Integrated Geographic Encoding and Referencing system) data from the U.S. Census Bureau is a good option to get the street data. Visit U.S. Census Bureau for more information or proceed to the ESRI ArcData Web interface for direct downloading available data. Street data do have a lot of attributive information, but one will still need to perform a lot of manual data editing. The data coverage includes only the USA.

2) ESRI has a separate product – ESRI StreetMap Premium, which provides access to the street data in SDC format both for North America and Europe. This product can be used in the Network Analyst directly and no data processing is required. Be aware that these network data are for read-only use and no edits to data can be applied. This data product provides the best performance as well as robust functionality, yet the least customization flexibility, because one can only use the street data as it is.

3) Navteq Navstreets and TeleAtlas Multinet are one of the most often used products for network analysis purposes. The data is delivered in MapInfo or ESRI Shape format and need to be processed in order to be used with the Network Analyst extension. Navteq does provide Navstreets data in the compiled ESRI File Geodatabase from the Q1 2011 yet some data processing still will be required. Depending on how much information you want to include into your network dataset, you will need to perform some joins and geometry processing operations as well as use ArcGIS geoprocessing tools or scripting language, for instance, Python. The data coverage includes the whole world.

4) OpenStreetMap – a collaborative Web project that aims at creating a free digital world map. One can download all the data including, but not limited to, street network data (http://downloads.cloudmade.com/). Street data do not have much attributive information so one would need to perform a lot of manual data editing. The data coverage includes the whole world.

Resources:

What is Network Analyst

TeleAtlas

Navteq Network for Developers

OpenStreetMap

OpenLR™

Further resources:

What’s new in ArcGIS 10

What’s new in ArcGIS Desktop 10

ESRI ArcGIS Desktop Blog

After browsing through topics related to the user experience in ArcGIS 10 on ESRI forums, I thought I could summarize some of the new features that are mentioned by many users. They are as below:

Q: When working on the construction of a feature, a mini toolbar Feature Construction follows me wherever I move the mouse cursor while digitizing a feature.

A: You have some alternatives here (the same applies to Edit vertices mini toolbar too):

a) You can move the mini toolbar a bit by pressing the Tab key.

b) You can press the Shift-Tab key combination in order to temporarily hide the mini toolbar.

c) You can switch off using mini toolbars from the Editor toolbar > Options > General tab > Show mini toolbars.

Q: Where has the Set Selectable Layers from the Selection menu option gone?

A: You can still use this option which is accessible now via the Customize menu. Go to the Customize > Customize Mode > Commands tab. Under the Selection category, you will find Set Selectable Layers command, which you can drag and drop into the Selection main menu in ArcMap.

Q: Where has the Annotation toolbar gone?

A: In ArcGIS 10, all the construction tools used for managing annotations are available in the Create Features window. Now one gets access to the Annotation Construction mini toolbar for working with annotation features.

I want users to pay attention to another simple, yet very useful, feature when working with geoprocessing in ArcMap. Using geoprocessing tools that do not produce any output feature classes implies that all changes are applying directly to the underlying feature layer. In this case, before running a geoprocessing tool you might like to start an edit session, so that you can undo the edits applied if required.

When digitizing a feature via using the Trace tool, keep in mind that you can use an offset which you can set up from the Trace options dialog box. Offsets can be both negative and positive numbers which would stipulate on which side of the traced feature the newly created feature will be located.

When having a feature selected on the map, you will notice the Selection Chip button that will appear near the selected feature. By clicking this button, you can discover what other features this feature is coincident with.

I have noticed too that a lot of users do not consider using the Topology toolbar, as they thing that it is used merely for checking the features’ geometries relationships and consistency. The Topology toolbar is a great timesaver when editing or constructing new features. Please have a go at using this toolbar, you’ll love it!

1.      Open a .csv file in Microsoft Excel 2007 by double-clicking it in the Windows Explorer.

2.      Right-click the first row and choose Insert>Entire row, if your .csv file does not have fields defined in the first row. Then add the string with the delimited field names, which have a symbol with which your data is delimited, as a first row. Note that you do not have split the field names row into the cells).

So you can get a row like this:

ID,Street,Address,X,Y

1,StreetA,24,5645645,97644355

2,StreetB,42,5675848,97546633

…

3323,StreetX,54,5743534,97454355

3.      Keep in mind that MS Excel has some limitations, including the maximum number of rows that a worksheet may contain – 1,048,576 rows. So, if your .csv file is larger you will have to split this file into several smaller ones or use some other software to prepare the delimited text file for import into ArcGIS. You may find it useful to use such applications as Notepad++ or Notepad2, which are capable of dealing with text delimited files.

4.      Save the file as a new .csv file, for example, Processed_<Filename>.csv and then close Excel without saving the newly created .csv file (click No in the coming dialog box).

5.      Create a new blank workbook in Excel. Click Data > Get External Data > From Text. Choose the file created in the previous step, for example, Processed_<Filename>.csv.

6.      Choose the Windows (ANSI) encoding or as appropriate. Start importing at row 1. As a delimiter, use the symbol your data is delimited with. Accept defaults for other options or specify what data type your fields shall get. Click Finish and import data in a first cell, as suggested by default.

7.      Save the file as an .xlsx file, for instance, ExcelDataFile_<number>.xslx, and then close Excel.

8.      Start ArcCatalog. Navigate to the folder where the newly created .xlsx file was created. Select the .xlsx file and expand it in the Catalog tree. Right-click the spreadsheet where the data reside, and choose Create feature class > From XY table. NB: if you have Data Interoperability extension enabled in ArcCatalog then ArcGIS treats an MS Excel file as an Interoperability dataset and you will not be able to convert your table to a feature class.

9.      Keep in mind that pretty often your X and Y fields should be used inversely. That is, in the dialog box, as X field, choose Y column of the .xlsx workbook. Similarly, as Y field, choose X column. Optionally, choose the coordinate system in which you want to define your features. Be aware, though, that you can leave the coordinate system undefined and specify a coordinate system after your feature class will be created. Choose the name for the feature class as required.

10.   Depending on how large the .csv file is and what hardware resources you have at your disposal, the processing may take from some seconds with several rows in the file to several hours when having a huge number of rows. When having the ArcCatalog processing the data, the application can go to the Not Responding state, which is a very common occurrence. Do not try to accelerate the process of data processing by moving ArcCatalog window or clicking Alt-Tab to see the job window, since it may slow down the process. If you are unsure whether the application is already hanged up, you can check the Memory use of the ArcCatalog in the Windows Task Manager. If this value is changing over time, then you can be nearly sure that the application is still working and you should not end this process.

11.   After the data load is done, start ArcMap and add the feature layer in order to check that the data has been imported properly.