fastmap.io BETA
docs sdk

Fastmap docs

Fastmap is a distributed drop-in replacement for map. As a rule of thumb, it usually runs faster than the map builtin when execution would otherwise take more than one second. Fastmap can run both on the cloud via fastmap.io's cloud service and on your local machine via multiprocessing. For more about fastmap's features, see the FAQ.

installation

pip install fastmap

quickstart

Fastmap can run both locally and on fastmap.io's cloud service.

"""Local example"""

import fastmap

fastmap_config = fastmap.init(exec_policy="LOCAL")

def sqrt(x):
    return x ** .5

results = list(fastmap_config.fastmap(sqrt, range(1000)))

To run using the cloud service, first obtain an API Token. Treat this like a password. Do not commit it to version control! Failure to do so could result in man-in-the-middle attacks or your credits being used by others (e.g. cryptocurrency miners). Then, try:

"""Cloud example"""

import fastmap
from config import FASTMAP_TOKEN

fastmap_config = fastmap.init(exec_policy="CLOUD",
                              secret=FASTMAP_TOKEN,
                              confirm_charges=True)

def sqrt(x):
    return x ** 0.5

results = list(fastmap_config.fastmap(sqrt, range(1000)))

Fastmap may print a message saying that it didn't make sense to use fastmap here. This is true. Fastmap is useful for heavy computations and the square root of a number is one of the lightest computations possible.

interface

Fastmap has exactly three public functions:

Additionally, three namespaces are available: And one custom exception:

fastmap.global_init(secret=None, verbosity="NORMAL", exec_policy="ADAPTIVE", confirm_charges=False, local_processes=None)
fastmap.init(secret=None, verbosity="NORMAL", exec_policy="ADAPTIVE", confirm_charges=False, local_processes=None)

Initialize fastmap. Using init returns a FastmapConfig object that you can call fastmap on. Using global_init allows you to use the fastmap method directly.

Parameters:
  • secret - The API token generated on fastmap.io. Treat this like a password. Do not commit it to version control! Failure to do so could result in man-in-the-middle attacks or your credits being used by others (e.g. cryptocurrency miners). If None, fastmap will run locally regardless of exec_policy.
  • verbosity: - "QUIET", "NORMAL", or "LOUD". Default is "NORMAL". Consider using the Verbosity namespace.
  • exec_policy: - "LOCAL", "CLOUD", or "ADAPTIVE". Default is "ADAPTIVE". To understand how this works, see the ExecPolicy namespace.
  • confirm_charges: - Manually confirm cloud charges. Default is False.
  • local_processes: - How many local processes to start. By default, fastmap will start a maximum of num_cores * 2 - 2.

Example global_init usage:

import fastmap

fastmap.global_init(exec_policy="LOCAL")
results = list(fastmap.fastmap(func, iterable))

Example init usage:

import fastmap

config = fastmap.init(exec_policy="LOCAL")
results = list(config.fastmap(func, iterable))

fastmap.fastmap(func, iterable, return_type="ELEMENTS", label=None)
FastmapConfig.fastmap(func, iterable, return_type="ELEMENTS", label=None)

Map a function over an iterable and return the results. By default, .fastmap is a generator and is executed lazily. To get the results as a sequence, wrap this in a list().

Parameters:
  • func - Function to map against. This must be stateles and cannot access the network or the filesystem. This function may call other functions but the entire execution tree must be limited to the file where fastmap is instantiated.
  • iterable - This may be either a sequence (list, tuple, etc.) or a generator.
  • return_type - "ELEMENTS" or "BATCHES". Default is "ELEMENTS". For more on how this works, see the ReturnType namespace.
  • label - Optional. Arbitrary string to track this execution and differentiate it from others on the dashboard. Only meaningful if some execution occurs on the cloud. This value is not consequential to the functionality of fastmap.

fastmap.FastmapConfig

Returned from the init function. Used when it is preferable to pass around a configuration object over a global configuration. Do not instantiate this directly.

Methods:
  • fastmap - Identical to the global fastmap function

fastmap.ExecPolicy

Convenience namespace class for the exec_policy argument on init and global_init.

Attributes:
  • LOCAL - Local processing does not require an API token and utilizes multiple processes on the host machine. It is similar to multiprocessing.Pool with a few additional advantages. Namely, fastmap handles generators and performs just-in-time computation. The local execution policy does not utilize the fastmap.io cloud service and is therefore always free.
  • CLOUD - The cloud execution policy will offload code to the fastmap.io cloud service and will minimize its use of local processing power. This is helpful if you have a backend server that you would otherwise like to keep performant for other tasks. This execution policy will consume fastmap credits but by default, you will get the option to confirm.
  • ADAPTIVE - The adaptive execution policy smartly distributes your job across both local and cloud resources. When running under the this policty, if the fastmap algorithm determines that it wouldn't make sense to run your code in the cloud, it won't and you won't be charged. Like the CLOUD policy, by default, you will get the option to confirm charges before anything is done. The ADAPTIVE policy is the default option and will usually be the fastest.

fastmap.ReturnType

Convenience namespace class for the return_type argument on the fastmap function.

Attributes:
  • ELEMENTS - The default. Using this causes fastmap to yield one element at a time. More intuitive but might be slower for certain workloads.
  • BATCHES - Causes fastmap to yield one batch at a time. Batches are what fastmap uses internally to divide work between multiple processes / machines. This eliminates the step of yielding individual batch elements and therefore, in isolation, causes fastmap to run marginally faster. Overall, however, this will only be faster if your code can work with batches directly.

fastmap.Verbosity

Convenience namespace class for the verbosity argument on init and global_init.

Attributes:
  • QUIET - "QUIET"
  • NORMAL - "NORMAL"
  • LOUD - "LOUD"

fastmap.EveryProcessDead

Fastmap will throw this exception if every process (worker), both locally and in the cloud, dies. If there is something wrong with your code, chances are good that this is the exception fastmap will throw.