Final Report After Defence

Master Project Report

A Framework for Integrating GIS Web Functions for Transportation Applications

Prepared by: Boyuan Guan, Master Student Department of Civil and Environmental Engineering Florida International University Miami, Florida 33174 Email: [email protected]

March 2009

Abstract

In the past decade, transportation, like all the other fields, has undergone a tremendous change accompanying the continuous development of IT technology. Tools like Google, Yahoo, and other search engines, Google Map, Microsoft office, and ESRI Geographic Information System (GIS) products have become an indispensible part of transportation engineering and planning. More and more powerful online tools have been developed to help transportation analyst to handle different kinds of tasks such as data sharing, online analysis, data viewing over satellite images, and online video analysis. However, most of these tools are developed to serve a specific purpose and are not compatible with each other. Therefore, transportation planners currently face the challenge of how to effectively manage these tools and integrate their functions into a single system. A Web GIS system designed to integrate as many functions as possible into a single platform can potentially solve the problem. For this purpose, a new method aimed at designing a framework for integrating various GIS Web Functions for Transportation Applications is being explored in this study. The framework should be designed to enable convenient transportation data sharing and online data analysis as well as providing useful interfaces of other Web GIS System such as searching, KML file exporting, satellite images, Street View, live traffic, online video, etc. A new pattern to design transportation web GIS system by using up-to-date technology like spatial geo-database, tiling and caching map, ArcGIS Server JavaScript extension, Google Maps’ APIs, Visual Earth APIs, AJAX, etc. will be introduced and discussed in this report. Key words: Web GIS, ArcGIS Server, JavaScript Extension, AJAX, Web Service, Transportation Data Sharing, Google Maps’ APIs, Visual Earth APIs

TABLE OF CONTENTS 1 Introduction.................................................................................................................................1 Background...................................................................................................... 1 Problem Statement..........................................................................................2 1.3 Objective....................................................................................................4

2 Methodologies..............................................................................................................................5 2.1 Discussion of Possible Solutions.................................................................5 2.2 Framework Design.....................................................................................5

3 System Enhancements and Realization....................................................................................8 3.1 Server-Side Design Functions.....................................................................8 3.2 Client-Side Enhancement.........................................................................14 3.3 Communication between the Server and Client Sides.............................21

........................................................................................................................................................22 4 User Applications......................................................................................................................23 4.1 Original Map Data with Metadata Download ...........................................23 4.2 Geo-database and Shape Files with Multiple Coordinate Systems Download ...................................................................................................... 25 4.3 KML File Export........................................................................................26 4.4 Verification of Area Types and Geometric Conditions .............................26 4.5 Street View Function................................................................................28 4.6 Inter-System Locating..............................................................................28 4.7 Geocoding Function.................................................................................30 4.8 Live Traffic............................................................................................... 30

5 Summary and Conclusions......................................................................................................32 6 Future Work..............................................................................................................................34 6.1 New Competitor in the GIS Arena: Microsoft............................................34 6.2 Reusable Components Design and Web Services....................................35

References.....................................................................................................................................36

1 Introduction Background The various software applications useful in analyzing and modeling transportation systems require the management of a huge amount of different spatial data (e.g., traffic data, land use data, social economic data, network data, etc.). As a tool to acquire, store, and process spatial data, Geographic Information System (GIS) applications can provide these capacities for transportation analysis. An application tool in an internet server environment, combined with GIS, can be used to disseminate such spatial information to dispersed users effectively, efficiently, and in a timely manner. Because both the data and GIS analysis software can be stored on the server, web-based solution for Transportation GIS is now widely accepted and has become prevalent in many transportation fields. The most prominent advantage of a Transportation Web GIS is the convenience of sharing and displaying data. It also allows online GIS analysis features like “intersect,” “buffer,” “measure,” “indentify,” “retrieve attribute data,” and so on. All of these relieve the transportation analyst from the inconvenience of having to access data through ArcGIS software since they are able to access the GIS application from their browsers at no cost. Traditional Transportation Web GIS design is based on an ArcGIS Server Framework and file system data access method of the operating system on a single server. Under this design pattern, the server takes almost all the load. Data access actions are based on operating system’s file system which is low efficient compare to database system. Therefore, the optimization of the Transportation Web GIS will be accomplished through the following steps: • • • • •

Store data in a geo-spatial database (SQL Server) through ArcSDE. Use tiling and caching map technology to reduce the time on generating images. Increase the speed by creating a personal website and using dynamic map source to reduce the load on the server. Use AJAX to improve the user experience. Use web services for public functions that can be calculated separately on another server.

Another approach to improve the performance of the Transportation Web GIS is to integrate functions from other commercial Web GIS like Google Map, Visual Earth. Functions from these commercial Web GIS Systems are known to be stable and to get results quickly. All of these functions can be realized through application programming interface (API) written in JavaScript, which work from the client side so that no extra load is added to the server. It also offers practical functions like satellite images, street views, live traffic information, and geo-coding, which is not easy to achieve through ArcGIS Server but can be achieved through JavaScript Extensions. As the above discussion of whole frame of Web GIS shows, a new design pattern which can integrating more Web GIS functions can indeed improve the performance of traditional Transportation Web GIS.

1

Problem Statement Web-based Transportation GIS [1] is a complex system that needs many different technologies such as database, web transmission, client side display, Geo-data manipulation program to realize. Generally speaking, a Web GIS is composed of four parts: • • • •

A database that can store both logical and special data like SQL Server, Oracle, or DB2. A server side program that responds to logical functions (basic page request like page reload, log in, log out, etc.) and GIS analysis functions (like display different layers, intersect, buffer, which client-side programs cannot handle). A client-side program that displays the results from server calculations, collects user requests and posts them back to the server, realizes the function of AJAX and Base Map API extensions like Google Maps’ APIs or Visual Earth’s APIs. A set of interfaces from other Web GIS that may provide public functions that are not easy to achieve through single small-scale GIS.

Although Web GIS has existed for almost a decade, some bottleneck problems still need to be solved, including: • • • •

The speed of the system. The requirement for different types of data format. The User-friendly level of interface. The reusability compatibility of public function components.

The speed of a Web GIS is restricted by three factors: the program speed on the application server, the transfer speed of the network, and the program speed on the client machine. The efficiency of the server-side program is always the first thing that should be considered by a framework designer. The most time consuming part of the server-side program is spatial data searching and manipulating. This means if the speed of reading and writing spatial data can be improved, the speed of whole system will be increased. The speed of the Web GIS is also determined by the mapping application; because the key feature of the Web GIS is the map displayed on the web page, this is an important bottleneck. In an ArcGIS framework, all the maps are published separately from the website, which means when a user request to open a web GIS system, the server should respond in two ways. One is to open the web page and the other to open the map, which is published on the server as a map service. It is very common that the map can be displayed after experiencing a long delay even after the web page has already been loaded. The more spatial data the map contains, the more processing time is needed to load the map. The automatic image generating process causes this phenomenon. For a regular map service, all the images displayed on the page will generate automatically by the specific query from the client side. The advantage of this feature is the flexibility in customizing the display styles; for example, users can decide which layers they want to be visible or not. The disadvantage of this feature, on the other side, is the time it takes to display the map. The solution for this problem is to create a cached map service consisting of tiled layers so that all the 2

images are pre-generated before the user request. Although the drawback to this solution is that not all the layers can be customized, it is worth using this alternative for the first “page load” action. When the users zoom to their study area, the program can change the map service feature. Because much less spatial data are generated when maps zoom in to a specific area, the speed of the service will not be affected that much. Current Web GIS programs face a second bottleneck in terms of the data variety in both displaying styles and data formats. For displaying styles, most of the existing systems only provide a specific map layer that contains predefined information like traffic signal information, intersection information, bus stop information, and so on. However, analysts may also need additional base information, like land use, geometric design, and facility type. These types of data may not need to be very precise but should cover every location of the study area. Integrating Google Satellite Map and Street View function through Google Map APIs and ArcGIS Server 9.3 JavaScript extension can help to provide these data. Another requirement in transportation analysis is the variety of data formats like shape files, layer files, geo database files, excel files etc., which all have different coordinate systems. The ArcGIS Server functions can be used to convert and coordinate these data formats. Another weak point is that the user interface is not user-friendly enough, despite the 2005 appearance of a revolutionary new technology called AJAX, which was first used and named by Google. This is not only a technology (actually, it is just a bundle of old technology with a new method), but a change in attitude towards designing online systems. The epiphany of this attitude was to separate the data communicating between the client side and the server side from the displayed page. In other words, when the user clicks the button or link (sends a request to server), the request should be sent to the server without any refresh or reload on the user’s page. This idea is widely accepted now after three years and many AJAX frameworks. Before ArcGIS 9.3 was released, ArcGIS Server has its own AJAX framework, which is very powerful. However, the AJAX framework may generate some conflict with the .NET framework when we develop our own program through the .NET platform. Thus, in the new 9.3 ArcGIS Server, the ESRI AJAX framework which was default framework in ArcGIS 2.0 becomes optional. Instead, the Microsoft AJAX framework called ALTAS is the default framework. This means a userfriendly interface can be developed by using ALTAS features like the drag/drop panel, auto complete dropdown list, and animate panel. The last problem that must be solved is how to manage the reusable function for a distributed system. Distributed Geographic Information System (DGIS) [2] is becoming a trend for the further development of Web GIS programs. To conform to this trend, designing our system into reusable components is reasonable and required. To do so, we used Web Service. Web Service is defined by the W3C as “a software system designed to support interoperable machine-tomachine interaction over a network”. To be more generally, Web services are published interfaces provided by other online systems which can realize different functionality requests from different users in different systems. In addition to the technologies discussed above, the other program trick is creating personal websites for users which can display only the study area for them instead of showing the statewide map data every time. Therefore, the amount of data need to be proceeded in the same 3

time are reduced and thus enhance the speed of the system. 1.3 Objective The overall objective of this research is to improve and enhance the performance of the Transportation Web GIS by using several different new technologies. Detail information of these technologies will be discussed in chapter 2. Specifically, the new framework can bring two major improvements for Web GIS. First, it increases the speed of displaying, processing, and data transferring of the system. Second, it extends the functions to satisfy more requirements that commonly appear during transportation analysis. Performance is a very important factor for a Web GIS, an aspect with plenty of room to enhance. The new framework will focus on data manipulation methods, map services management methods, map source retrieval methods, and functionality publishing methods to improve the performance. New technologies like spatial databases, cached map services, AJAX, Web Service, and “dynamic map source” as well as a few program tricks will be used in the new framework. After applying these technologies, maps will display much faster, the experience will be friendlier for the user, and data transaction time will be shorter. Functionality is already very powerful in the ArcGIS Server. However, some users are already getting used to the tools that commercial Web GIS like Google Map can provide. As summarized in Table 1.1, Google Map has several advantages compared to ArcGIS Server System. What is more, some functions like satellite image-based maps require a huge budget for data collection, which is almost impossible for any single transportation project. For these reasons, the functions of a traditional Web GIS need to be extended for transportation analysis. In the new framework, the extension is realized by using Google Maps’ APIs, new features from ArcGIS Server 9.3, and some APIs from other commercial systems. After applying these new functions, the system will have a base map with satellite images, an embodied search engine from Google Map, an embodied Street View from Google Map, and the accessibility for the whole set of interfaces provided by Google Table 1.1 Comparisons of Google Map and ArcGIS Server SPEED DATA FUNCTIONS Google Maps Fast Plentiful and Basic Functions Various ArcGIS Slow Precise Basic and Advanced GIS Server Functions

STABILITY Stable Unstable

4

2 Methodologies 2.1 Discussion of Possible Solutions As discussed in the previous section, four bottlenecks must be conquered to improve the Transportation Web GIS: the speed of the system; the different data formats; the userfriendliness of the interface; and the reusability of the public function component. There are several ways to enhance the speed of the spatial data accessing. Since the system will be built under the ArcGIS framework, the discussion below is based on the ArcGIS platform. First, like regular data, spatial data are stored in a hard drive in a specific format (.MXD, .LYR, etc.). Therefore, if the program must access spatial data, it should go through an interface from Arc Desktop to retrieve the data. In this way, access to all the data is based on file system that does not provide high speed. One solution for this problem is to upgrade to a database system. However, unlike logical data, spatial data cannot be stored directly into database system because they cannot be interpreted from attribute fields into tables. Thus, the appropriate solution is a geo-database system consisting of SQL Server 2000 and ArcSDE. Spatial data can be stored in the same database system with the logical data and no additional input/output (I/O) reading or writing actions are needed when accessing the spatial data. As a result, the speed of the whole system can be improved. Another unique method to enhance the performance of the Web GIS in the proposed framework is to cooperate with other commercial products like Google Map or Visual Earth. Nowadays, people are so accustomed to using Google Map that it is almost forgotten that this application appeared just four years ago. Its amazing success comes from its unbeatable features: fast, stable, huge data coverage, and so on. All of these features are very difficult to realize through a traditional Web GIS because of the restrictions on the server’s design limitations, budget for data, and management ability. Fortunately, both Google Map and Visual Earth provide a set of powerful APIs for users. What is more, ESRI provides a JavaScript extension for integrating Google Map and Visual Earth functions into the ArcGIS Server Web GIS. From these new technologies, programmer can reduce the server load by moving some functions from server side to client by using the Google Maps APIs or Visual Earth APIs. 2.2 Framework Design As a Web GIS system, functions can be basically divided into server-side and client-side functions. Figure 2.1 provides a workflow chart for the system. On the server side, geo-database, cached map services, and some programming tricks will apply to enhance the performance of processing the data. On the client side, a set of functions will be integrated through a public interface like Google Map APIs. Thus, many high performance and user-friendly functions will be added without increasing the server load. Beside these, AJAX technology will be used to optimize the data communication between server side and client side. Both the performance and the user experience will be increased for the data transfer from this design. To realize the design, several technical points need to be accomplished. Figure 2.2 lists all the technical points that help enhance the performance and extend the function of the system. After applying these technologies, the new framework will optimize the speed of accessing data, 5

retrieving spatial data, displaying maps, transferring data, as well as other functions.

Figure 2.1 Transportation Web GIS System Design

6

Figure2.2 Technologies Used in the Solution 7

3 System Enhancements and Realization The Transportation Web GIS System can be generally divided into three parts: • • •

Server-side design improvements including the data storage (logical and spatial), programming, and publishing (map services, web services, and website). Client-side design improvements including programming, integrating Google Map functions and Visual Earth functions throughout their APIs. Communication between client- and server-side programming through AJAX and Web Service.

As the design for these three parts of system are independent so they are discussed separately in the following sections. 3.1 Server-Side Design Functions On the server side, several optimization methods can improve data processing time including storing spatial data in geo-database through ArcSDE, tiling and caching map, site considerations, and reducing the load on server by using dynamic map sources. ArcSDE geo-databases allow the user to use other ESRI products like the ArcGIS Server in our system, as well as custom applications to store, use, and manage all the GIS data in one of the following commercial database management systems (DBMS): IBM DB2, IBM Informix, Microsoft SQL Server, or Oracle. Key tasks to set up a geo-database through ArcSDE include: • • • • •

Installing software: ArcSDE is a part of an ArcGIS Server when installed. It retrieves the data that communicates with the client server and database. It provides files for the setup and administration of the geo-database on the server. Configuring the DBMS: ArcSDE will configure the database on the server with the installation tools step by step. Creating user accounts: This feature controls access to the geo-database; user accounts with specific permissions need to be created appropriately. Creating connections to the geo-database: This feature works with the data; a connection to the geo-database from client applications needs to be created. Connections can be made directly between the application server, and the database. Maintaining the Geo-Database: As the data in the Geo-Database changes, the database administrator (DBA) performs tasks, such as compressing the database, updating statistics, performing backup and recovery procedures, and altering storage locations to maintain performance levels.

Another key point is to manage the permissions of both the logical and the spatial databases. This may be confusing because three security levels must be considered: the operating system, the logical database system, and the ArcGIS Server system. The assignment for the rights and roles are summarized in Table 3.1. 8

Table 3.1 ArcGIS Server Task and Role Assignment Task Management of physical database structures (data files, tables, and indexes) Management of logical database constructs (data models, rules, subtypes, networks, locators, and versions) Management of users and permissions • Database permissions • Object permissions Database backup and recovery Performance tuning • Of the database • Of workflows (how users utilize the GIS) Data distribution (replication/synchronization) Geo-database compression and updating statistics

Role DBA GIS administrator DBA (database permissions) Data owner (object permissions) DBA DBA (database) GIS administrator (workflows) GIS administrator GIS administrator

The other optimization method is to change the dynamic map to a cached map on the server. A server map cache is a collection of pre-rendered map tiles that can be used to display a map service. This allows a map service to quickly display maps because the map image does not have to be rendered simultaneously after receiving the client’s request. The cost of rendering the image is paid only once when the cache is created. The server map cache greatly improves performance because the map does not need to be computed each time the user requests a map from the server. Instead, the images are retrieved from the map cache for the requested map extent and resolution. This means that ArcGIS map services can scale to very large numbers of users and maps served every hour. As shown in Figures 3.1 and 3.2, only one service will start up in the ArcGIS Server rather than starting a service for every single user, thus reducing the load on the server side. There are also some tricks in programming that can help reduce the load and thus improve the speed of the system. One of the common methods is customizing maps from a large range to small study area, thus reducing the data handled significantly. In our system, this function is called Personal Website. By using this function, users can create their own page which inherit all the functions from main page but only load the data needed for the specific user. To realize this function, Geo-processing function in the ArcGIS Server was used. In general, to create a personal web site, 4 steps are needed which are including: 1. 2. 3. 4.

Register for a new account Select the data for the personal website Enter the information for the personal website Create web page and generate the GIS service

First of all, Users need to register first and thus the user account could be used as an identity for the personal page. After registered successfully, users can access a window show the state wide map data as shown in the figure 3.1. On this map, users are allowed to select the map data they needed to display in their personal website by county or by block group as shown in the figure 3.2. When selected specific map data, users are required to provide some information for 9

creating the main page of their personal website such as the title of the page, the descriptions, and the layers they want to be included in their map data which is illustrated in the figure 3.3. A personal website is created after this step. All of the personal websites will be listed on the main page of the whole system as shown in the figure 3.4. As the purpose this system is to share the data, so all the users have the right to view other users’ personal website and download their map data as shown in the figure 3.5. As shown in the figure 3.6, once the personal website creator login, more control can be access such like upload layers, delete layers, and move layers from one group to another group. Overall, from the personal website function, not only the performance of the system can be enhanced, but also the more widely data can be shared from different users. What is more, since the data is less than the main page and the page is only designed for a specific user, more customized function like upload layers, delete layers, and move layers are able to provide for the website creator.

Figure 3.1 Statewide Map Data

10

Figure 3.2 Select Personal Map Data

Figure 3.3 Input information for Personal Page

11

Figure 3.4 Personal Website List

Figure 3.5 Personal Website

12

Figure 3.6 Personal Website Login by Creator The major advantages for using personal website including: • • • •

Reduce the load of the main page Enhance the performance of the system Provide more resource to share from every single user Provide more powerful functions in personal website

Although using the personal website can improve the speed of the system effectively, the disadvantage is that the page will increase as the personal website grows. As a result, the load of the server will continue to rise. The optimal solution for this problem is using the dynamic map source so that all users can access only one page to view different map services as shown in figure 3.7. The most time consuming part, generating images, in the whole process can thus successfully avoided as shown in the figure 3.8

13

Dynamic Map Service

Figure 3.7 Dynamic Map Service Dynamic Map Services Work Flow: Client Request  Map Services  Generating Images (time consuming)  Display Map V.S Cached Map Services Work Flow: Client Request  Tiled Images (fast as picture)  Display Map Figure 3.8 Dynamic Map Service Work Flow 3.2 Client-Side Enhancement The client-side program should be responsible for more functionality to reduce the load on the server side as well as integrating new functions from other systems. The purpose is to cooperate with other commercial Web GIS applications like Google Map and Visual Earth by using the APIs they provide. For example, the Google Maps API lets us embed Google Maps in our own web pages with JavaScript. The API provides a number of utilities for manipulating maps (just like on the http://maps.google.com web page) and adding content to the map through a variety of services, allowing robust map applications to be created on a website. The Web GIS based on Google Maps’ API lack professional geo-analysis function compare to 14

ArcGIS Server Web GIS. However, integrating these two Web GIS applications will provide the best of both worlds and satisfied a variety of user demands. To realize this purpose, four steps must be considered: • •

• •

Coordinate system conversion Two types of map services connection Function integration Interface organization

Coordinate system conversion is the basic point to integrate two systems. The ArcGIS Serverbased Web GIS platform is very flexible in supporting system coordinates. From the newly released ArcGIS Server 9.3 documentation, more than 4000 Projected Coordinate Systems and more than 1000 Geographic Coordinate systems are built in. By contrast, the coordinate system used in Google Map or Visual Earth is very stiff. Google Maps and Microsoft Virtual Earth use a Mercator projection based on the 1984 World Geodetic System (WGS) geographic coordinate system (datum). This Mercator projection supports spheres only, unlike the ESRI Mercator implementation, which supports spheres and ellipsoids. To emulate the sphere-only Mercator, it is necessary to use a sphere-based geographic coordinate system (GCS) for the correct Mercator equations. This sphere-based geographic coordinate system is called the “WGS 1984 Major Auxiliary Sphere.” The WGS 1984 Web Mercator projected coordinate system and the WGS 1984 Major Auxiliary Sphere geographic coordinate system are now available in ArcGIS version 9.3. If the data are based on a different geographic coordinate system, such as the 1927 North American Datum (NAD27), ED 50, Amersfoort, or Tokyo; then misalignment may be quite noticeable at the local scale. Data in the 1983 North American Datum (NAD83), WGS 1984, or another geographic coordinate system (GCS) that use the GRS80 ellipsoid are usually close enough to align without correction except when zoomed in. Features of Google’s coordinate system are summarized below (Figures 3.9 and 3.10 illustrate these features in detail): • • • • •

Datum - WGS84. Projection - Mercator. Unit system - pixels. Axes - standard east, non-standard south. Origin - Near the North Pole on the International Date Line.

To match these two map services, all ArcGIS Server map layers must be projected into the WGS 1984 Web Mercator coordinate. The easiest way is change the coordinate system of the map frame in ArcMap to WGS 1984 Web Mercator. Thus, all the map layers will display with the same coordinate system as the one defined for the data frame, which is called “data projected on the fly.” The detailed steps are: • • •

Import all the layers into ArcMap. Check that all the map layers match automatically despite different coordinate systems. Change the coordinate system of the data frame: Projected Coordinate Systems  World  WGS 1984 Web Mercator. 15

Datum: WGS84

Projection: Web Mercator

Figure 3.9 Google Map's Datum and Projection

Datum: WGS84

Figure 3.10 Properties of WGS_1984_Web_Mercator After converting the coordinate system, the shape and layout of the map will change to match the new coordinate system as illustrated in Figure 3.11. After coordinate system conversion, all 16

Projection: Web Mercator

ArcGIS map layers will distort to match Google Map coordinates. Figure 3.12 shows a sample screen shot for the system.

Figure 3.11 Map Distort After Conversion

17

Overlay after coordinate system conversion

Figure 3.12 Integrating of ArcGIS Map and Google Map Map in GIS is a kind of spatial data, defined as “data or information that identifies the geographic location of features and boundaries on Earth, such as natural or constructed features, oceans, and more. Spatial data is usually stored as coordinates and topology, and is data that can be mapped. Spatial data is often accessed, manipulated or analyzed through Geographic Information Systems (GIS)”. After converting the map, the topology becomes uniform. The next task is to link two map services by the coordinates. Although two maps share the same coordinate system, to cater to worldwide usage, the parameters in Google Maps’ API require latitude and longitude information; the WGS 1984 Web Mercator uses meters. This is an impediment for linking two kinds of map services because the coordinates are the only connection between two map services. The solution for this problem is using a method provided by ArcGIS Server 9.3 which can automatically projects one map on the coordinate system that used in the other map so that can get the coordinate of the same location which can be read in the other map. Figure 3.13 illustrates this function as written in VB language. ''' <param name="xSP"> ''' input x coordinate of the point '''

18

''' <param name="ySP"> ''' input y coordinate of the point ''' ''' <param name="opType"> ''' "0":Convert from Meter to LatLong ''' ''' <param name="opType"> ''' "1":Convert from LatLong to Meter ''' ''' ''' a convert point ''' Private Function MeterLatLong(ByVal xSP As Double, ByVal ySP As Double, ByVal opType As String) Dim pSpRef1 As ESRI.ArcGIS.Geometry.ISpatialReference Dim pSpRFc As New ESRI.ArcGIS.Geometry.SpatialReferenceEnvironment Dim pPCS As ESRI.ArcGIS.Geometry.IProjectedCoordinateSystem pPCS = pSpRFc.CreateProjectedCoordinateSystem(102113) pSpRef1 = pPCS pSpRef1.SetFalseOriginAndUnits(0, 0, 1000000) pSpRef1.Changed() Dim pGCS As ESRI.ArcGIS.Geometry.IGeographicCoordinateSystem Dim pSpRef2 As ESRI.ArcGIS.Geometry.ISpatialReference pGCS = pSpRFc.CreateGeographicCoordinateSystem (ESRI.ArcGIS.Geometry.esriSRGeoCSType.esriSRGeoCS_WGS1984) pSpRef2 = pGCS pSpRef2.SetFalseOriginAndUnits(0, 0, 1000) pSpRef2.Changed() Dim pPoint As ESRI.ArcGIS.Geometry.IPoint = New ESRI.ArcGIS.Geometry.PointClass pPoint.PutCoords(xSP, ySP) Dim pGeo As ESRI.ArcGIS.Geometry.IGeometry pGeo = pPoint If opType = "0" Then pGeo.SpatialReference = pSpRef1 pGeo.Project(pSpRef2) ElseIf opType = "1" Then pGeo.SpatialReference = pSpRef2 pGeo.Project(pSpRef1) End If Return pPoint End Function

Figure 3.13 The Function to Convert Coordinate Systems By using this function, two kinds of map services can cooperate successfully, including autoadjust, Inter-system location, and function integration. With auto-adjust, for example, although 19

two sets of maps are already matched to each other, once a user requests the map layer to be “spanned,” the other map cannot respond to the same request. As discussed above, the functions can be realized by transferring the coordinates between different systems as coordinate become a media between two systems. The detailed procedure is: • • • • • •

• •

Google Map responds to the user request. Client side program (JavaScript) gets the new coordinates. AJAX function sends the coordinates back to the server side. Server side program (C#) handles the coordinates on the server. Server side program (C#) move the ArcGIS Server hosted Map to the same center location. Server side program (C#) matches the two maps again. AJAX function gets the processed result from server side and sends it back to client side. Client side program (JavaScript) displays the processed map.

Figure 3.14 shows this procedure with screen capture.

Auto Align

Figure 3.14 Auto-Align of Two Maps 20

After this step, the only problem left is how to organize these functions from different systems. Fortunately, all the Google Map or Visual Earth function access their JavaScript APIs and almost all the ArcGIS Server functions are designed through server program written in C# or VB. These features allow us to design all the functions on the same page, and organize them separately. The detailed organization is shown in Figure 3.15.

ArcGIS Server Functions

Figure 3.15 Interface Organization

Google Map Functions

After these steps, ArcGIS Server Web GIS and Google Maps are integrated both in terms of displaying maps and GIS functionality. The performance of the new system is enhanced because: • • • •

Reductions in the server load due to moving some functions to the client side. Faster and more stable functions from Google Map are implanted into system. Some analysis tasks can be performed easily on Google Map. Google Map works as a default base map when the ArcGIS Server Map is being transmitted.

3.3 Communication between the Server and Client Sides Server- and client-side communication is also a critical part of the whole system. Almost every action of the ArcGIS Server function will need a post back action of the page. If designed in a traditional web mode, a single span function will cause the page to reload three times. Every reload will refresh the map; assuming the loading time is one minute, the user must wait three minutes for the map to reload even without network delay. AJAX [3] is an important web development model for browser-based web applications, and is helpful for communicating between the client and server sides. The AJAX model uses several technologies that come together to create a powerful new model, through widely-used and well-known technologies like XML JavaScript, HTTP and XHTML. Google first used and named this new powerful browserbased model in their high performing online GIS application: Google Maps. Web Services [4], another technology that often works together with AJAX, are self-contained, self-describing, and modular. Unlike earlier, more tightly-coupled distributed object approaches, like Common Objects Request Brokers (CORBA), Web Service systems support an XML message-centric approach, allowing us to build loosely coupled, highly distributed systems that span organizations. Web Services also generalize many of the desirable characteristics of Web 21

GIS, such as standards for providing general-purpose specifications for publishing, locating, and invoking services across the Web. Web Services also use widely-used and well-known technologies such as XML and HTTP, just as AJAX does. Since AJAX and Web Services are XML-based structures, they are able to leverage each other’s strength. Figure 3.16 shows the steps needed to optimize the client / server (C/S) communication model design with these two technologies. The steps needed to realize the design are: • • •

Create as many Web Services as possible in single or multiple servers for public functions. Create AJAX-enabled framework in the client side program. Design the server-side program in reusable components.

Figure 3.16 Steps to Realize Integration

22

4 User Applications Web GIS for transportation has certain characteristics that identify it from other Web GIS designs. Transportation analysts require certain features that help determine these characteristics. For general transportation analysis, four parts should be taking into consideration: transportation operations, transportation planning, transportation safety, and ITS applications. The demands for all of these four areas are based in a common set of data. Although the details of the data may vary among different subjects—for example, an analyst examining operations may focus more on traffic data like AADT, intersection information, or signal information, while an analyst in planning may need socioeconomic data—all must share and obtain data from the Web GIS. For this reason, a Transportation Web GIS need focus more on data sharing compared to other Web GIS designs in terms of the original logical and spatial data as well as metadata, data format converting, coordinate system converting, and spatially processed data. Besides this common demand, these four different study areas also have unique demands for the system. In the new enhanced Transportation Web GIS, a much wider range of demands can be satisfied. The overall demand is summarized into three categories: data sharing, online geo-analysis, and functions from other commercial Web GIS databases. Figure 4.1 shows more detail about the application of this information. 4.1 Original Map Data with Metadata Download Data is the foundation for all kinds of transportation analysis. A more efficient data sharing system will enhance the performance of the transportation analyst. In the FSUTMS GISonline 2.0 system, all the data are divided into four categories: Traffic Data, Network Data, Socioeconomic Data, and Boundaries, which cover almost the entire requirement for transportation analysis. Users are able to preview all the data online before they download them. They may also customize the data to their own study area online.

Datum: WGS84

Metadata is a very important part of GIS data. It helps to understand the database by providing information like the coordinate system used for the dataset, the definition for the variables, the creation date, the purpose behind creating the dataset, etc. However, the metadata is currently the weakest point for Transportation GIS data. Different data are distributed from different providers without a standard format for metadata; some metadata are too simple to understand and thus make the data very hard to understand. In our system, the metadata format will correspond to the FDOT format, designed in a technology called XLST [6], so that the user can get enough support information with the data. Figure 4.2 shows the function panel for downloading and the uniform format for meta-data.

23

Projection: Web Mercator

Figure 4.1 Users' Applications Summary

24

Figure 4.2 Data Downloading with Meta-data 4.2 Geo-database and Shape Files with Multiple Coordinate Systems Download In the traditional GIS system, the data sharing function would normally just provide data with their origin. The users must convert the data to formats consist with their systems. On the other hand, get a set of data with a Geo-Database is another requirement from the user because this will save time if they want to store the data in ArcSDE. To shore up this weak point regarding data sharing function diversity, all the map data can be converted to different coordinate systems 25

prior to download. Figure 4.3 shows the function panel for downloading function.

Figure 4.3 Data Converting and Downloading Panel 4.3 KML File Export GIS data sharing is always limited because the tools that can open the data are very rare, and most of them are very big and expensive. Providing a KML file download can help the user solve this problem because they may review the map and attribute table on their local computer with Google Earth, available for free on the Internet. Figure 4.4 shows an example of the effect of a KML displaying in Google earth. 4.4 Verification of Area Types and Geometric Conditions Verification of area types and the survey of land use activities surrounding transportation facilities and verification of geometric conditions such as roadway alignments and number of lanes are two common requirements for traffic operation, safety analysis, and traffic planning. Figures 4.5 and 4.6 are the examples about how to realize these functions in FSUTMS GIS Online System.

26

Figure 4.4 KML File in Google Earth

Figure 4.5 Area Types Verification

27

Figure 4.6 Geometric Conditions Verification

4.5 Street View Function This function allows users to pop up a street view window to show the location where they clicked after enable the street view function. From the new popped window, users can get information like: • • • •

Geometry design. Pavement condition. Traffic Single and Traffic Sign’s Condition. Facility Type.

Figure 4.7 shows an example of a pop-up window with street view that was obtained by clicking the study location. 4.6 Inter-System Locating This function helps users to continue their work on other Web GIS. In this system, Google Map, one of the most widely used commercial Web GIS, illustrates this function. In addition, the intermedia between two different GIS models is the coordinate of the location that the user clicked, which is the foundation of all kinds of GIS databases. In other words, this function can be realized between any two or among several different GIS programs only if the systems can support all or part of others’ coordinate systems. Figure 4.8 shows an example of switching the work from an ArcGIS server system to Google Map in the same location.

28

Figure 4.7 Street View Window

Figure 4.8 Inter-System Locating

29

4.7 Geocoding Function The geocoding function in this framework is designed by using the Google Maps APIs because they are faster and more stable. The distance will be computed follow the formula below. Figure 4.9 shows an example of a simple geocoding using the formula shown in Figure 4.10.

Figure 4.9 Search by Address Formula and Program Used to Compute Distance between Two Locations: protected double getDistance(double lat1, double lng1, double lat2, double lng2) { double dbDistance = 2*Math.Asin(Math.Sqrt(Math.Pow((Math.Sin( (lat1-lat2)/2)),2)+Math.Cos(lat1)*Math.Cos(lat2) *Math.Pow((Math.Sin((lng1-lng2)/2)),2)))*dbRadium; dbDistance = dbDistance/dbMile; return dbDistance; }

Figure 4.10 Inter-System Locating 4.8 Live Traffic 30

Live traffic can be a supplement for traffic when the user needs some traffic analysis like traffic control design. This kind of data was very hard to get and not easy to display on the page. However, with Google Maps APIs, the user can easily compare live traffic data with their own GIS data. Figure 4.11 shows an example of live traffic data.

Figure 4.10 Live Traffic Data from Google Map

31

5 Summary and Conclusions This study of this report has accomplished the following major research tasks: •

• • • •

An integrated framework for implementing various GIS functions in transportation applications is developed. An enhanced GIS server configuration design is accomplished. A solution on improving performance of data communication between client and server side is developed. A compatible method to integrate commercial web GIS functions is developed. A high performance client design is accomplished.

As discussed above, an integrated framework for implementing various GIS functions in transportation applications involve three design aspects: server-side design, client-side design, and the design for the communication protocol between the server and client sides. The framework is designed based on ArcGIS Server framework and several new components including ArcSDE, Microsoft Atlas, AJAX.net, Google Map functions from Google Maps APIs, Microsoft Visual Earth (VE) function and APIs, and multiple data sharing functions. Compared to the traditional framework provided by the ArcGIS Server, we propose an enhanced application with regard to data sharing functions, satellite images base map, street view functions, multiple Geo-Searching functions, data customization, and online live traffic information, etc. These improvements extend the functions of current transportation analysis programs in their limited abilities to a wide range of data sets based on commercial web GIS like Google Maps and Visual Earth. Enhancing the function allows users to compare the data they want with real satellite images, locating the study area the same way they use Google Maps, downloading the data with many different formats, and even making their own web page for their study area. The new designs for the proposed framework can be realized through the following steps: •

•

•

•

Reduce data access time. Change the file system-based data management method to ArcSDE. Spatial data can be stored in the same database system with the logical data and no additional I/O reading and writing actions are required when accessing spatial data. As a result, the speed of the whole system will increase. Reduce the time needed to access a map by caching the map services. Reduce the time to generate tiled images when the user requests the map or a specific location on the map. Twenty zoom levels will be predefined and all the tiled images for these levels will be generated and stored on the server. Improve the user experience by using AJAX to improve the refresh protocol, which will improve the speed of the system. Not all of the page will be posted back to the server, but only a request or the relevant data will be send back to the server and the rest of the page will be unchanged so that the user can continue to work undisturbed by refresh time. This will enhance the speed of the system because only part of the page will be refreshed instead of reloading the whole page. Reduce the load on the server by moving some functions to the client side. Rich client side is a trend for web developing. The load on each side should be balanced to optimize system performance. The map data does not need to be reloaded for every single user 32

•

•

•

event. An image with the coordinate system will allow users to work more comfortably. Implant faster and more stable functions from commercial Web GIS System like Google Maps and Visual Earth. Some functions from commercial Web GIS System are faster, more stable, and more user-friendly like Geo-Searching, satellite images, Street View, and Live Traffic. What is more, these functions are widely used by professional engineers and the APIs are free of charge. Implanting these functions can improve the performance of the system and reduce the learning curve for users. Helps users to continue their analysis from local GIS to Google Map by using Intersystem locating function. The Google Maps application is becoming an essential tool for all kinds of research. The inter-system locating function will help users to switch their work from one system to another. Actually, this function is just an interface—it is essentially an illustration. The function can be realized among all kinds of GIS applications if they provide the coordinate interface. Reduce the wait time by using Google Maps as a base map when the ArcGIS Server Map is under transmission. Although waiting is unavoidable even after using map-caching technology, the blank time makes the user uncomfortable when they work on the map. Providing a base map is an acceptable alternative when the ArcGIS Server Map is under transmission.

This framework presents a different way to design a Web GIS, but it is still not a fully developed product. Integrating more functions from different frameworks and different systems can improve the performance of the Web GIS System to make it more efficient. Three aspects of Web GIS design are enhanced with other technology in order to satisfy more requirements. The major contributions of this framework include: • • • • • •

Increasing the speed of accessing data Enhancing the performance of the system Enhancing the user experience through the user-friendly interface Reducing the developing period Reducing the load on the server Providing more functions

33

6 Future Work 6.1 New Competitor in the GIS Arena: Microsoft In the foreseeable future, changes in the market will change the way GIS will develop and how users will work with GIS. Compared to five years ago, the method for working with Web GIS has changed significantly with the appearance of ArcGIS Server solution and Google Maps. ESRI and Google have occupied the highest position in the Web GIS arena until now. However, competition from other companies may break this pattern. One of the greatest competitors will be Microsoft. As the leader in IT area for so many years, Microsoft always can find its way to grasp the market in all kinds of IT technology. Recently, Microsoft has realized the importance of Web GIS and is getting ready to publish his Microsoft GIS and Web GIS solution: Visual Earth + Map Point + SQL Server 2008. Compared to other solutions, the Microsoft application has two major advantages: •

•

To avoid potential compatibility problems, it integrates all the developing functions in current Microsoft products: .NET + Visual Earth + Map Point + Microsoft Office + SQL Server. This integration is a great attraction for both the developer and the GIS analyst because they don’t have to switch their work among several different platforms and worrying about the compatibility of their work. Users of Visual Studio, SQL Server, and Microsoft Office may prefer to use products from same company. As the .NET framework is becoming more and more popular, the growing quantity of .NET users is huge enough to change the current market pattern.

There are already several mature products from Visual that show great competitiveness in the Web GIS field in the following areas: • • • • • • • •

Financial Services Hospitality & Travel Manufacturing Media & Entertainment Oil & Gas / Utilities Public Sector Real Estate Retail

For detailed information see: http://www.microsoft.com/virtualearth/industry/ In order to make this product competitive in the market, developers should keep an eye on these changes in the Web GIS arena and try the best to make the GIS application easy to migrate from one framework to another.

34

6.2 Reusable Components Design and Web Services This part of the task is not easy to realize because currently not every single ArcGIS Server System can afford the cost of a distributed system considering the cost of the server is still an economic burden. However, as trends in hardware show that the server will become more and more powerful as it becomes less and less expensive. It is true that improvements are limited if we create Web Services in a single server even if the server has multiple central processing units (CPU). However, if there are several servers that respond to data management, map services, and geo-analysis separately, the effect and performance will be significantly increased. The day that the price of the servers will not be a problem will come along with the enhanced power of web services as described in this paper.

35

References 1. Transportation Web GIS System: http://www.gis.fhwa.dot.gov/ . (Accessed 01/19/2009) 2. DIGS: http://www.dhpc.adelaide.edu.au/projects/dgis/ . (Accessed 01/19/2009) 3. AJAX: Asynchronous JavaScript and XML: http://en.wikipedia.org/wiki/AJAX . (Accessed 01/19/2009) 4. Web Service: A 'Web service' (also Web Service) is defined by the W3C as "a software system designed to support interoperable machine-to-machine interaction over a network".Web services are frequently just Web APIs that can be accessed over a network, such as the Internet, and executed on a remote system hosting the requested services. 5. Coordinate Systems supported by ESRI’s ArcGIS products: Projected Coordinate Systems and Well Known IDs (>4000): http://resources.esri.com/help/9.3/arcgisserver/apis/javascript/arcgis/help/jshelp/pcs.htm . Geographic Coordinate Systems and Well Known IDs (>1000): http://resources.esri.com/help/9.3/arcgisserver/apis/javascript/arcgis/help/jshelp/gcs.htm (Accessed 01/22/2009)

.

5. XSLT: a language for transforming XML documents into other XML documents. http://www.w3.org/TR/xslt (Accessed 02/13/2009)

36