Reports

The bottleneck is that the entire GeoJSON data is being sent to the browser and rendered by Folium's JS library, for large datasets this is very slow.

You can help this by simplifying the geometry (reduce the complexity of the polygons) and tiling the data (break the GeoJSON into smaller tiles and load them on demand. Try using Vector tiles as they are a more efficient file format and have a library that supports it.

#1: The first thing to do is reduce the complexity of the polygon geometries. This will reduce the size of the GeoJSON data which inturn will help with performance.

import streamlit as st
import geopandas as gpd
import folium
from streamlit_folium import st_folium

st.set_page_config(layout="wide")

@st.cache_data
def load_and_simplify_shp(tolerance=0.01):
    shp_df = gpd.read_file('testlayer.shp')
    simplified_df = shp_df.simplify(tolerance=tolerance)
    return simplified_df.to_json()

layer_to_add = load_and_simplify_shp()

with st.form(key='mymap'):
    m = folium.Map(location=[31.5943, -102.8927], zoom_start=12)
    folium.GeoJson(layer_to_add, name="my layer", style_function=lambda feature: {
        "fillColor": "yellow",
        "color": "black",
        "weight": 0.5,
        "fillOpacity": 0.4
    }).add_to(m)

    st_folium(m, height=1000, use_container_width=True, key='map')

    submit_button = st.form_submit_button("Submit")

2: Tile the Data

For larger sets, you can tile the GeoJSON data. This involves breaking the data into smaller tiles and loading them on demand as the user zooms and pans the map. This is trickier but I can provide an example if you want to explore that option.

3: Vector Tiles

Vector tiles are a better way to represent geographic data(IMO). They are smaller than GeoJSON and can render much faster. Libraries like ipyleaflet support vector tiles. This is the trickiest of the three, but also has the most performance improvement.

Implementation Instructions

Replace your load_shp function with load_and_simplify_shp. Adjust the tolerance parameter to find a balance between performance and detail. Test the performance and see if it meets your needs.

Notes: Simplification can result in loss of detail, so it's good to find a tolerance that has a good balance between performance and accuracy. Tiling and Vector tiles require additional libraries and a deeper understanding of web mapping.(I am just really learning this myself) If you would like to explore tiling or vector tiles, let me know, and I can dig deeper into some detailed code and explanations. I hope this helps, have fun with it!

also this is a good link to check out: https://granulate.io/blog/java-profiling-3-key-areas-profile-2/ it gives a pretty good rundown on some concepts you may find suitable for your project.

79529854