So, while working with caching and scrapping, I understood the difference between immutable and mutable objects/datatypes very clearly. I had a scenario, where I am webscraping an API, the code looks like this.

from aiocache import cached

@cached(ttl=7200)
async def get_forecast(station_id: str) -> list[dict]:
    data: dict = await scrape_weather(station_id)
    # doing some operation
    return forecasts

and then using this utility tool in the endpoint.

async def get_forecast_by_city(
    param: Annotated[StationIDQuery, Query()],
) -> list[UpcomingForecast]:
    forecasts_dict: list[dict] = await get_forecast(param.station_id)
    forecasts_dict.reversed()

    forecasts: deque[UpcomingForecast] = deque([])
    for forecast in forecasts_dict:
        date_delta: int = (
            date.fromisoformat(forecast["forecast_date"]) - date.today()
        ).days
        if date_delta <= 0:
            break
        forecasts.appendleft(UpcomingForecast.model_validate(forecast))

    return list(forecasts)

But, here is the gotcha, something I was doing inherently wrong. Lists in python are mutable objects. So, reversing the list modifies the list in place, without creating a new reference of the list. My initial approach was to do this

for forecast in forecasts_dict.reversed():
  ...

This didn’t work due to the exact same reason. But, this shouldn’t affect the endpoint, right? As each call would get a new reference…??? Turns out, NO! The call to the utility function returns the same underlying reference due to the caching mechanism.

In my above example, I was getting a blank list on every 2nd call, because,

  1. call 1: reverses the list -> prints the data properly
  2. call 2: reverses the already reversed list -> early return -> blank list
  3. call 3: reverses the same list again -> doesn’t match condition -> get proper data

So, what’s the best solution here? Making sure that the reference of a list never gets mutated in place without making a copy of it. So, to solve the problem, I used

for forecast in reversed(forecasts_dict):
  ...

This creates a new iterator, which is iterated over by the for loop. Really, learnt an awesome thing today!