Data Drudgery & Beautiful Betas: Mining Flow Maps Update

Previous posts on my Porcupine Gold Rush International Flow-Mapping project here:

A Graduate Fellow Introduction

Maps and Mining: Getting Data Out of ArcMap & the Beginning of Flow Mapping

Part I: Data Drudgery

I know that I am hardly the first Sherman Fellow to lament mind-numbing data management which becomes inevitable at a certain point in a mapping project’s life-cycle. Having successfully sorted out how to make flow maps in Arc, I now just need a complete database to plug in to the software. That means getting enough entries into my “Porcupine Inventory” of moved people, objects, and ideas to make it work. What does this mean on a practical level? In Chapter 1 of my dissertation I argue that the Cobalt silver rush in Northern Ontario (a predecessor to the Porcupine) was an part of an international mining network. One of my supporting pieces of evidence is the fact that the Otago School of Mines regularly sent graduates to work in Northern Ontario – Cobalt was considered a very prestigious placing.  The information from the Otago School of Mines comes from a 14 May 1910 clipping in the Otago Daily Times “Otago School of Mines Annual Report.”

final

I then create a database entry like this:

Name: Norman Fisher
Date: 14 May 1910
How_many: 1
Type: Person
Point_of_origin: Otago, New Zealand
End_point: Cobalt, Ontario
Instigator: Temiskaming Mining Company
Source: “Otago School of Mines Annual Report,” Otago Daily Times, 14 May 1910, 5
Notes: Mine Manager trained at Otago School of Mines

And repeat for ad infinitum as I go through my primary documents. Thrilling stuff.

Part II: Beautiful Betas: Blueshift Map Builder & Ethnic Origin of 1921 Porcupine Residents

As a way of counteracting some of the drudgery, I’ve continued to experiment with different flow-mapping platforms. By far the most exciting has been Blueshift, a simple flow-map browser application which allows you to make beautiful flow maps in just a few minutes.  Since the embed function doesn’t work yet (it is still in the beta phase) I took a short screen capture to show how it works.  You can also visit the interactive map to zoom around and play for yourself.

The data used here is census information on Porcupine residents’ “place of origin” in 1921. Here’s the original data expressed as a pie, for the curious.

originpie1921

It is hugely problematic for a lot of reasons all on its own – it is unclear if individuals were assigned ethnicity by the census-taker or whether they self-identified their origin (most likely the former). The data also obscures that fact that many people, regardless of how they were identified ethnically, had come from the US. Obviously it also obscures indigenous residents. It thus must be taken with a large grain of salt.

However, I found that Blueshift forced me to make additional oversimplifications I didn’t like. So, for example, the original data lists origins like: “French,” “Scandinavian,” “Hebrew,” “Negro” and “Chinese and Japanese.” Obviously this is somewhat of a mixed bag of continents, nations, and ethnic groups. However, I could only sort my data by city, state, or country in Blueshift. So for “China and Japan,” I had to pick one (I chose China), and I likewise just had to pick a representative country for broad categories like “Scandinavia” and “Negro.” Furthermore, the “world” baselayer uses modern national boundaries and so the application, like many mapping apps, simply does not jive well with historic and/or “messy” data.

On the plus side, it is very attractive. In fact the finished product looks exactly like I imagined my final flow map looking, only better. So I see this as a viable alternative to hard-to-learn programs like ArcMap and QGIS and I think it is even easier and more functional that ArcGIS Online.

Mica Jorgenson is a PhD candidate in the Department of History at McMaster University. She studies the environmental history of gold rushes around the globe. You can find her on twitter: @mica_amy

Posted in Blog Tagged with: , , , ,

Leave a Reply

Your email address will not be published. Required fields are marked *

*

mcmaster logo
Made possible by a generous gift from the Lewis & Ruth Sherman Foundation.

wpgform-debug.php::37::$_SERVER

Array
(
    [SERVER_SOFTWARE] => Apache/2.4.23 (Debian)
    [REQUEST_URI] => /data-drudgery-beautiful-betas-mining-flow-maps-update/
    [REDIRECT_STATUS] => 200
    [HTTP_HOST] => scds.ca
    [HTTP_ACCEPT_ENCODING] => x-gzip, gzip, deflate
    [HTTP_USER_AGENT] => CCBot/2.0 (http://commoncrawl.org/faq/)
    [HTTP_ACCEPT] => text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
    [HTTP_X_FORWARDED_FOR] => 54.197.150.255
    [HTTP_X_FORWARDED_HOST] => scds.ca
    [HTTP_X_FORWARDED_SERVER] => erasmus.mcmaster.ca
    [HTTP_CONNECTION] => Keep-Alive
    [PATH] => /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
    [SERVER_SIGNATURE] => 
Apache/2.4.23 (Debian) Server at scds.ca Port 80
[SERVER_NAME] => scds.ca [SERVER_ADDR] => 192.168.122.12 [SERVER_PORT] => 80 [REMOTE_ADDR] => 192.168.122.1 [DOCUMENT_ROOT] => /var/www/wordpress/current [REQUEST_SCHEME] => http [CONTEXT_PREFIX] => [CONTEXT_DOCUMENT_ROOT] => /var/www/wordpress/current [SERVER_ADMIN] => webmaster@localhost [SCRIPT_FILENAME] => /var/www/wordpress/current/index.php [REMOTE_PORT] => 38773 [REDIRECT_URL] => /data-drudgery-beautiful-betas-mining-flow-maps-update/ [GATEWAY_INTERFACE] => CGI/1.1 [SERVER_PROTOCOL] => HTTP/1.1 [REQUEST_METHOD] => GET [QUERY_STRING] => [SCRIPT_NAME] => /index.php [PHP_SELF] => /index.php [REQUEST_TIME_FLOAT] => 1490242491.153 [REQUEST_TIME] => 1490242491 )

wpgform-debug.php::39::$_ENV

Array
(
)

wpgform-debug.php::41::$_POST

Array
(
)

wpgform-debug.php::43::$_GET

Array
(
)

wpgform-debug.php::45::locale

en_US
C

wpgform-debug.php::51::$wp_filter['init']

WP_Hook Object
(
    [callbacks] => Array
        (
            [0] => Array
                (
                    [create_initial_post_types] => Array
                        (
                            [function] => create_initial_post_types
                            [accepted_args] => 1
                        )

                    [create_initial_taxonomies] => Array
                        (
                            [function] => create_initial_taxonomies
                            [accepted_args] => 1
                        )

                    [000000006af9fc9f00007f35ef1a566cwp_security_plugin_init] => Array
                        (
                            [function] => Array
                                (
                                    [0] => AIO_WP_Security Object
                                        (
                                            [version] => 4.1.8
                                            [db_version] => 1.8
                                            [plugin_url] => http://scds.ca/wp-content/plugins/all-in-one-wp-security-and-firewall
                                            [plugin_path] => /var/www/wordpress/releases/20130426203411/wp-content/plugins/all-in-one-wp-security-and-firewall
                                            [configs] => AIOWPSecurity_Config Object
                                                (
                                                    [configs] => Array
                                                        (
                                                            [aiowps_remove_wp_generator_meta_info] => 
                                                            [aiowps_prevent_hotlinking] => 
                                                            [aiowps_enable_login_lockdown] => 1
                                                            [aiowps_allow_unlock_requests] => 
                                                            [aiowps_max_login_attempts] => 3
                                                            [aiowps_retry_time_period] => 5
                                                            [aiowps_lockout_time_length] => 60
                                                            [aiowps_set_generic_login_msg] => 
                                                            [aiowps_enable_email_notify] => 
                                                            [aiowps_email_address] => john.fink@gmail.com
                                                            [aiowps_enable_forced_logout] => 
                                                            [aiowps_logout_time_period] => 60
                                                            [aiowps_enable_invalid_username_lockdown] => 
                                                            [aiowps_unlock_request_secret_key] => 2su0vetsvth235jvy112
                                                            [aiowps_enable_whitelisting] => 
                                                            [aiowps_allowed_ip_addresses] => 
                                                            [aiowps_enable_login_captcha] => 
                                                            [aiowps_captcha_secret_key] => dwjpr6vqpdgusp4w6686
                                                            [aiowps_enable_manual_registration_approval] => 
                                                            [aiowps_enable_registration_page_captcha] => 
                                                            [aiowps_enable_random_prefix] => 
                                                            [aiowps_enable_automated_backups] => 1
                                                            [aiowps_db_backup_frequency] => 4
                                                            [aiowps_db_backup_interval] => 2
                                                            [aiowps_backup_files_stored] => 2
                                                            [aiowps_send_backup_email_address] => 1
                                                            [aiowps_backup_email_address] => john.fink@gmail.com
                                                            [aiowps_disable_file_editing] => 
                                                            [aiowps_prevent_default_wp_file_access] => 
                                                            [aiowps_system_log_file] => error_log
                                                            [aiowps_enable_blacklisting] => 
                                                            [aiowps_banned_ip_addresses] => 
                                                            [aiowps_enable_basic_firewall] => 
                                                            [aiowps_enable_pingback_firewall] => 
                                                            [aiowps_disable_index_views] => 
                                                            [aiowps_disable_trace_and_track] => 
                                                            [aiowps_forbid_proxy_comments] => 
                                                            [aiowps_deny_bad_query_strings] => 
                                                            [aiowps_advanced_char_string_filter] => 
                                                            [aiowps_enable_5g_firewall] => 
                                                            [aiowps_enable_brute_force_attack_prevention] => 
                                                            [aiowps_brute_force_secret_word] => 
                                                            [aiowps_cookie_based_brute_force_redirect_url] => http://127.0.0.1
                                                            [aiowps_brute_force_attack_prevention_pw_protected_exception] => 
                                                            [aiowps_brute_force_attack_prevention_ajax_exception] => 
                                                            [aiowps_enable_404_logging] => 
                                                            [aiowps_enable_404_IP_lockout] => 
                                                            [aiowps_404_lockout_time_length] => 60
                                                            [aiowps_404_lock_redirect_url] => http://127.0.0.1
                                                            [aiowps_enable_rename_login_page] => 
                                                            [aiowps_site_lockout] => 
                                                            [aiowps_site_lockout_msg] => 
                                                            [aiowps_enable_spambot_blocking] => 
                                                            [aiowps_enable_comment_captcha] => 
                                                            [aiowps_enable_automated_fcd_scan] => 
                                                            [aiowps_fcd_scan_frequency] => 4
                                                            [aiowps_fcd_scan_interval] => 2
                                                            [aiowps_fcd_exclude_filetypes] => 
                                                            [aiowps_fcd_exclude_files] => 
                                                            [aiowps_send_fcd_scan_email] => 
                                                            [aiowps_fcd_scan_email_address] => john.fink@gmail.com
                                                            [aiowps_fcds_change_detected] => 
                                                            [aiowps_copy_protection] => 
                                                            [aiowps_last_backup_time] => 2017-03-10 20:53:49
                                                            [aiowps_enable_debug] => 
                                                            [aiowps_enable_custom_login_captcha] => 
                                                            [aiowps_block_debug_log_file_access] => 
                                                            [aiowps_enable_custom_rules] => 
                                                            [aiowps_custom_rules] => 
                                                            [aiowps_enable_login_honeypot] => 
                                                            [aiowps_cookie_brute_test] => 
                                                            [aiowps_enable_autoblock_spam_ip] => 
                                                            [aiowps_spam_ip_min_comments_block] => 
                                                            [aiowps_prevent_site_display_inside_frame] => 
                                                            [aiowps_prevent_users_enumeration] => 
                                                        )

                                                    [message_stack] => stdClass Object
                                                        (
                                                        )

                                                )

                                            [admin_init] => 
                                            [debug_logger] => AIOWPSecurity_Logger Object
                                                (
                                                    [log_folder_path] => /var/www/wordpress/releases/20130426203411/wp-content/plugins/all-in-one-wp-security-and-firewall/logs
                                                    [default_log_file] => wp-security-log.txt
                                                    [default_log_file_cron] => wp-security-log-cron-job.txt
                                                    [debug_enabled] => 
                                                    [debug_status] => Array
                                                        (
                                                            [0] => SUCCESS
                                                            [1] => STATUS
                                                            [2] => NOTICE
                                                            [3] => WARNING
                                                            [4] => FAILURE
                                                            [5] => CRITICAL
                                                        )

                                                    [section_break_marker] => 
----------------------------------------------------------


                                                    [log_reset_marker] => -------- Log File Reset --------

                                                )

                                            [cron_handler] => AIOWPSecurity_Cronjob_Handler Object
                                                (
                                                )

                                            [user_login_obj] => AIOWPSecurity_User_Login Object
                                                (
                                                    [key_login_msg] => aiowps_login_msg_id
                                                )

                                            [user_registration_obj] => AIOWPSecurity_User_Registration Object
                                                (
                                                )

                                            [backup_obj] => AIOWPSecurity_Backup Object
                                                (
                                                    [last_backup_file_name] => 
                                                    [last_backup_file_path] => 
                                                    [last_backup_file_dir_multisite] => 
                                                )

                                            [scan_obj] => AIOWPSecurity_Scan Object
                                                (
                                                )

                                            [captcha_obj] => AIOWPSecurity_Captcha Object
                                                (
                                                )

                                        )

                                    [1] => wp_security_plugin_init
                                )

                            [accepted_args] => 1
                        )

                    [wpgform_debug] => Array
                        (
                            [function] => wpgform_debug
                            [accepted_args] => 1
                        )

                )

            [1] => Array
                (
                    [wp_widgets_init] => Array
                        (
                            [function] => wp_widgets_init
                            [accepted_args] => 1
                        )

                )

            [5] => Array
                (
                    [smilies_init] => Array
                        (
                            [function] => smilies_init
                            [accepted_args] => 1
                        )

                )

            [10] => Array
                (
                    [wp_cron] => Array
                        (
                            [function] => wp_cron
                            [accepted_args] => 1
                        )

                    [_show_post_preview] => Array
                        (
                            [function] => _show_post_preview
                            [accepted_args] => 1
                        )

                    [rest_api_init] => Array
                        (
                            [function] => rest_api_init
                            [accepted_args] => 1
                        )

                    [kses_init] => Array
                        (
                            [function] => kses_init
                            [accepted_args] => 1
                        )

                    [wp_schedule_update_checks] => Array
                        (
                            [function] => wp_schedule_update_checks
                            [accepted_args] => 1
                        )

                    [wpGForm::ProcessGoogleForm] => Array
                        (
                            [function] => Array
                                (
                                    [0] => wpGForm
                                    [1] => ProcessGoogleForm
                                )

                            [accepted_args] => 1
                        )

                    [wpgform_register_post_types] => Array
                        (
                            [function] => wpgform_register_post_types
                            [accepted_args] => 1
                        )

                    [wpgform_init] => Array
                        (
                            [function] => wpgform_init
                            [accepted_args] => 1
                        )

                    [000000006af9fc3000007f35ef1a566cinit] => Array
                        (
                            [function] => Array
                                (
                                    [0] => TGM_Plugin_Activation Object
                                        (
                                            [plugins] => Array
                                                (
                                                )

                                            [parent_menu_slug] => themes.php
                                            [parent_url_slug] => themes.php
                                            [menu] => install-required-plugins
                                            [default_path] => 
                                            [has_notices] => 1
                                            [is_automatic] => 
                                            [message] => 
                                            [strings] => Array
                                                (
                                                    [page_title] => Install Required Plugins
                                                    [menu_title] => Install Plugins
                                                    [installing] => Installing Plugin: %s
                                                    [oops] => Something went wrong.
                                                    [notice_can_install_required] => Array
                                                        (
                                                            [0] => This theme requires the following plugin: %1$s.
                                                            [1] => This theme requires the following plugins: %1$s.
                                                            [singular] => This theme requires the following plugin: %1$s.
                                                            [plural] => This theme requires the following plugins: %1$s.
                                                            [context] => 
                                                            [domain] => 
                                                        )

                                                    [notice_can_install_recommended] => Array
                                                        (
                                                            [0] => This theme recommends the following plugin: %1$s.
                                                            [1] => This theme recommends the following plugins: %1$s.
                                                            [singular] => This theme recommends the following plugin: %1$s.
                                                            [plural] => This theme recommends the following plugins: %1$s.
                                                            [context] => 
                                                            [domain] => 
                                                        )

                                                    [notice_cannot_install] => Array
                                                        (
                                                            [0] => Sorry, but you do not have the correct permissions to install the %s plugin. Contact the administrator of this site for help on getting the plugin installed.
                                                            [1] => Sorry, but you do not have the correct permissions to install the %s plugins. Contact the administrator of this site for help on getting the plugins installed.
                                                            [singular] => Sorry, but you do not have the correct permissions to install the %s plugin. Contact the administrator of this site for help on getting the plugin installed.
                                                            [plural] => Sorry, but you do not have the correct permissions to install the %s plugins. Contact the administrator of this site for help on getting the plugins installed.
                                                            [context] => 
                                                            [domain] => 
                                                        )

                                                    [notice_can_activate_required] => Array
                                                        (
                                                            [0] => The following required plugin is currently inactive: %1$s.
                                                            [1] => The following required plugins are currently inactive: %1$s.
                                                            [singular] => The following required plugin is currently inactive: %1$s.
                                                            [plural] => The following required plugins are currently inactive: %1$s.
                                                            [context] => 
                                                            [domain] => 
                                                        )

                                                    [notice_can_activate_recommended] => Array
                                                        (
                                                            [0] => The following recommended plugin is currently inactive: %1$s.
                                                            [1] => The following recommended plugins are currently inactive: %1$s.
                                                            [singular] => The following recommended plugin is currently inactive: %1$s.
                                                            [plural] => The following recommended plugins are currently inactive: %1$s.
                                                            [context] => 
                                                            [domain] => 
                                                        )

                                                    [notice_cannot_activate] => Array
                                                        (
                                                            [0] => Sorry, but you do not have the correct permissions to activate the %s plugin. Contact the administrator of this site for help on getting the plugin activated.
                                                            [1] => Sorry, but you do not have the correct permissions to activate the %s plugins. Contact the administrator of this site for help on getting the plugins activated.
                                                            [singular] => Sorry, but you do not have the correct permissions to activate the %s plugin. Contact the administrator of this site for help on getting the plugin activated.
                                                            [plural] => Sorry, but you do not have the correct permissions to activate the %s plugins. Contact the administrator of this site for help on getting the plugins activated.
                                                            [context] => 
                                                            [domain] => 
                                                        )

                                                    [notice_ask_to_update] => Array
                                                        (
                                                            [0] => The following plugin needs to be updated to its latest version to ensure maximum compatibility with this theme: %1$s.
                                                            [1] => The following plugins need to be updated to their latest version to ensure maximum compatibility with this theme: %1$s.
                                                            [singular] => The following plugin needs to be updated to its latest version to ensure maximum compatibility with this theme: %1$s.
                                                            [plural] => The following plugins need to be updated to their latest version to ensure maximum compatibility with this theme: %1$s.
                                                            [context] => 
                                                            [domain] => 
                                                        )

                                                    [notice_cannot_update] => Array
                                                        (
                                                            [0] => Sorry, but you do not have the correct permissions to update the %s plugin. Contact the administrator of this site for help on getting the plugin updated.
                                                            [1] => Sorry, but you do not have the correct permissions to update the %s plugins. Contact the administrator of this site for help on getting the plugins updated.
                                                            [singular] => Sorry, but you do not have the correct permissions to update the %s plugin. Contact the administrator of this site for help on getting the plugin updated.
                                                            [plural] => Sorry, but you do not have the correct permissions to update the %s plugins. Contact the administrator of this site for help on getting the plugins updated.
                                                            [context] => 
                                                            [domain] => 
                                                        )

                                                    [install_link] => Array
                                                        (
                                                            [0] => Begin installing plugin
                                                            [1] => Begin installing plugins
                                                            [singular] => Begin installing plugin
                                                            [plural] => Begin installing plugins
                                                            [context] => 
                                                            [domain] => 
                                                        )

                                                    [activate_link] => Array
                                                        (
                                                            [0] => Activate installed plugin
                                                            [1] => Activate installed plugins
                                                            [singular] => Activate installed plugin
                                                            [plural] => Activate installed plugins
                                                            [context] => 
                                                            [domain] => 
                                                        )

                                                    [return] => Return to Required Plugins Installer
                                                    [dashboard] => Return to the dashboard
                                                    [plugin_activated] => Plugin activated successfully.
                                                    [activated_successfully] => The following plugin was activated successfully:
                                                    [complete] => All plugins installed and activated successfully. %1$s
                                                    [dismiss] => Dismiss this notice
                                                )

                                        )

                                    [1] => init
                                )

                            [accepted_args] => 1
                        )

                )

            [15] => Array
                (
                    [GoogleSitemapGeneratorLoader::Enable] => Array
                        (
                            [function] => Array
                                (
                                    [0] => GoogleSitemapGeneratorLoader
                                    [1] => Enable
                                )

                            [accepted_args] => 0
                        )

                )

            [99] => Array
                (
                    [check_theme_switched] => Array
                        (
                            [function] => check_theme_switched
                            [accepted_args] => 1
                        )

                )

        )

    [iterations:WP_Hook:private] => Array
        (
            [0] => Array
                (
                    [0] => 0
                    [1] => 1
                    [2] => 5
                    [3] => 10
                    [4] => 15
                    [5] => 99
                )

        )

    [current_priority:WP_Hook:private] => Array
        (
            [0] => 0
        )

    [nesting_level:WP_Hook:private] => 1
    [doing_action:WP_Hook:private] => 1
)

wpgform-debug.php::56::$wp_filter['template_redirect']

WP_Hook Object
(
    [callbacks] => Array
        (
            [0] => Array
                (
                    [_wp_admin_bar_init] => Array
                        (
                            [function] => _wp_admin_bar_init
                            [accepted_args] => 1
                        )

                )

            [1] => Array
                (
                    [GoogleSitemapGeneratorLoader::DoTemplateRedirect] => Array
                        (
                            [function] => Array
                                (
                                    [0] => GoogleSitemapGeneratorLoader
                                    [1] => DoTemplateRedirect
                                )

                            [accepted_args] => 0
                        )

                )

            [10] => Array
                (
                    [wp_old_slug_redirect] => Array
                        (
                            [function] => wp_old_slug_redirect
                            [accepted_args] => 1
                        )

                    [redirect_canonical] => Array
                        (
                            [function] => redirect_canonical
                            [accepted_args] => 1
                        )

                    [kpg_permalink_finder] => Array
                        (
                            [function] => kpg_permalink_finder
                            [accepted_args] => 1
                        )

                    [responsive_content_width] => Array
                        (
                            [function] => responsive_content_width
                            [accepted_args] => 1
                        )

                )

            [11] => Array
                (
                    [rest_output_link_header] => Array
                        (
                            [function] => rest_output_link_header
                            [accepted_args] => 0
                        )

                    [wp_shortlink_header] => Array
                        (
                            [function] => wp_shortlink_header
                            [accepted_args] => 0
                        )

                )

            [1000] => Array
                (
                    [wp_redirect_admin_locations] => Array
                        (
                            [function] => wp_redirect_admin_locations
                            [accepted_args] => 1
                        )

                )

        )

    [iterations:WP_Hook:private] => Array
        (
        )

    [current_priority:WP_Hook:private] => Array
        (
        )

    [nesting_level:WP_Hook:private] => 0
    [doing_action:WP_Hook:private] => 
)

wpgform-core.php::1725::ProcessGoogleForm

Array
(
)