Modelbit ships.
Performance week! We took the opportunity to work on various scalability and performance improvements last week. In particular, Modelbit is now more scalable when working with very large batches, e.g. from Snowflake. And we made various scalability and performance improvements to the model registry, especially for customers with a lot of models who make a lot of updates.
⚒️ New Features
The logs viewer now links to the Endpoint, if any, that was used to receive the request.
🐛 Bug Fixes
Fixed a bug where the Keep Warm confirmation would sometimes flicker an incorrect confirmation before showing the correct confirmation
Introducing the new Deployments homepage! Fit for many more deployments, complete with tagging; deployment search; and filtering by tag or execution status. Learn more.
⚒️ New Features
Improved the "Add Deployment" wizard for users who've already deployed at least once via Git.
Added a confirmation to Keep Warm that is clearer about which version of the deploying is being kept warm.
Added more detail to the Keep Warm tooltip text clarifying how it's billed.
Added a tooltip clarifying that the usage data export always exports all data regardless of selected filters.
Improved the Git sync performance for large workspaces with large repos.
🐛 Bug Fixes
Fixed the tooltip on "Add Deployment" on protected branches.
Fixed a bug where the client-side Git validation was overly strict about YAML syntax in
metadata.yaml.
⚒️ New Features
Added the
requirements_txt_path=parameter tomodelbit.deployfor when you want to specify exact Python requirements. Learn moreSimplified some of the requirements of the
modelbitPython package so that it can work in a broader variety of environments.Improved the error message for GPU inferences that time out while waiting in the queue.
Improved the error message when schemaVersion is wrong in a training job's metdata.yaml.
Added helpful guidance to environment-build error messages about GCC being missing.
🐛 Bug Fixes
Fixed a bug where the usage chart would cut off the legend if there were too many deployments in it.
Fixed a bug where selected deployments do not appear checked in the appropriate dropdown in usage charts.
Fixed a bug where the logs viewer did not properly render newlines when
\rcharacters were included.Fixed a bug where calling
modelbit.add_jobwould eliminate any pre-existing schedules for that job.Fixed a bug where Python exceptions were not handled correctly in Python 3.9 training jobs.
Fixed a bug where first-time users auto-joining a workspace that has been migrated to a new cluster would get a 500 error.
Fixed a bug issue where multiple Git repo backups running concurrently could cause the web app to crash.
⚒️ New Features
The Training Jobs logs viewer now shows all log lines even when there are a lot of them.
Streamlined the UX for A/B testing with Endpoints.
Extended
modelbit.debug()with additional recent error information.When logging in with the
modelbit.login()link, the resulting web page now shows an error if the specified branch is missing.The logs viewer now shows which Endpoint, if any, was called to call the deployment.
Made various improvements to autoscaling algorithms and host assignment algorithms.
Updated the Git-based new-user flow to always ask before uploading a pre-existing SSH key.
Cleaned up the error message when calling
modelbit.get_dataset()with an invalid dataset name inside a deployment.
🐛 Bug Fixes
Fixed a bug where training jobs would not redeploy when common files they depend on were updated.
Fixed a bug where calling `logger.log` would crash training jobs.
Fixed a bug where deleted datasets were still accessible to running deployments.
Fixed a bug where it could look like the dataset schema refresh button wasn't doing anything because it didn't clear the old schema.
Fixed a bug where helpful error text in the logs viewer could sometimes render in the wrong font.
Fixed a bug where Modelbit's periodic Git backups would prevent updates while backing up.
⚒️ New Features
Updated the Model registry UX to align the graphs with their corresponding model details.
Made
regiona required parameter inget_inference(...)to reduce user confusion.By popular demand, made Notebooks once again the default FTUE in the getting-started wizard.
Training Jobs no longer show runs older than a month old.
🐛 Bug Fixes
Fixed a bug where Modelbit would confusingly save and resurrect the weights of inactive A/B tests in Endpoints.
Fixed multiple syntax errors in the example
get_inference(...)code for DataFrame-mode deployments.Fixed a bug where expanding folders in the model registry could cause the UX to jitter.
Fixed a bug where Slack alerts would not trigger when the Python Exceptions that trigger them are very long.
Worked around a regression in Hex that caused Modelbit to stop correctly capturing inference code that contains newlines.
⚒️ New Features
Grouped DataFrame-mode batch inference requests together in the logs viewer.
Modelbit no longer routes requests to hosts that are still booting, thus improving effective cold-start performance.
"Auto redeploy when packages are changed" now only redeploys deployments that depend on those specific packages.
Slack alerts now re-alert each time the alert count crosses an order-of-magnitude threshold.
Usage charts now store which deployments are selected in the URL, to facilitate saving and sharing.
Added units to usage-and-latency chart tooltips for clarity.
modelbit package addandmodelbit.add_package(...)no longer try to create packages when no package exists.Made model names wrap instead of truncating in the model registry.
Made the private package screen easier to understand for workspaces with lots of private packages.
Made error messages more helpful when inference functions return values that aren't JSON-serializable.
Made it clearer in the Run Job UX that string arguments must be double-quoted.
Streamlined the API-login-with-link flow to reduce confusion.
CLI validation errors now link to docs in more cases.
Added additional protections against "thundering herd" outages.
🐛 Bug Fixes
Fixed a bug where endpoints would wait for shadow deployments to complete before returning results.
Fixed a bug where copying truncated `curl` requests could copy spurious spacing characters.
Fixed a bug where certain types of timeouts would show the wrong error message in the logs viewer.
Fixed a bug where warnings in the logs viewer could eagerly include actual log lines in the warning UX.
⚒️ New Features
Improved the request chunking in
mb.get_inference(...)to eliminate occasionalRateLimitExceedederrors that could occur with large, wide inputs.Added automatic retry logic to
mb.get_inference(...).Added a warning when DataFrame-mode deployments are called with extra columns not used by the deployment.
Improved the memory performance of handling large batches of logs.
🐛 Bug Fixes
Fixed a bug where the logs viewer would sometimes erroneously render
Noneasnull.Fixed a bug where deleting training jobs did not terminate running instances of that job.
Fixed a bug where the training jobs log viewer would spaz out if there are just enough log lines to scroll the unexpanded viewer but not the expanded viewer.
⚒️ New Features
Made Snowpark deployments up to twice as fast at runtime with performance improvements to Modelbit's logging and instrumentation code!
Added a helpful message when deployments are redeployed because of changes to private packages or common files.
Clarified the error message when logs are omitted due to request rate.
Made the "Prints" section of the logs viewer always appear so that it's clearer that printouts are always logged.
Raised the request rate threshold for triggering dropped logs.
Added a loading animation to Training Jobs that are in the process of being added.
🐛 Bug Fixes
Fixed a bug where jobs that were scheduled but not yet run would show a "Next Run" in the past.
Fixed a bug where specifying
torchwith an environment qualifier would cause Modelbit to skip auto-adding the right CUDA drivers.Fixed a bug where implicitly switching branches to a branch that does not exist could cause a Python notebook to get stuck on the nonexistent branch.
Fixed a bug where a page reload was necessary to see the logs from a training job that just completed.
🏋️ Introducing Training Jobs 2.0! Training Jobs are now their own top-level Modelbit object, complete with their own runtime environments, their own section in the top nav of the product, and a revamped API and UX. Learn more.
⚒️ New Features
Modelbit now allows small hidden files in the Git repo, to play nicer with static analysis tools, linters, IDE preference files, etc.
Added the optional
skip_extra_files_discoveryparameter tomb.deploy()andmb.add_job()for scenarios where you only want explicitly-specified local dependencies to be included in Modelbit.Improved the error message when the Snowflake connection isn't permitted to read any tables.
Improved the experience of copying log lines out of the logs viewer by automatically stripping the invisible characters used for spacing.
Updated the Content-Type header of async REST responses to be
application/json.Added
modelbit debugto the CLI which prints a long list of Python environment information which the team might ask you for when debugging support issues.Improved the caching performance of our CDN.
🐛 Bug Fixes
Fixed a bug where attempting to add common files located outside the repo would fail.
Fixed a bug where
modelbit.models()could occasionally fail with a not-very-helpful error message.Fixed a bug where attempting to view a nonexistent common file could get you stuck loading forever.
Fixed a bug where an extraneous copy button sometimes showed up in the logs viewer.
Fixed a bug where the model registry viewer would get stuck loading forever when the registry is empty.
Fixed a bug where the common files viewer could appear to lag while clicking through many common files on a slow connection.
⚒️ New Features
Added the
response_format,response_webhookandresponse_webhook_ignore_timeoutoptions to the Pythonget_inference(...)API.Added a button to generate a fresh Git remote deploy key.
Made performance improvements to the model registry for registries with many models.
Improved the readability of the SSH keys table in Settings.
Improved the error message when results sizes are too large to return over REST.
Improved the error message when calling a DataFrame-mode deployment with more than one parameter.
Improved the error message when a REST call times out.
🐛 Bug Fixes
Fixed a bug where
modelbit clonewould output "Clone Complete!" even when the clone terminated with an error.Fixed a bug where
delete_models(...)would fail when passed a very large number of model names.Fixed a bug where
get_model(...)would fail when run on a branch with a slash in its name.Fixed a bug where branches that only have archived deployments would not auto-expand that section.
Fixed some minor UX nits on the Environment tab.
🎉 New, improved, and way more helpful Modelbit docs! We have rebuilt our docs from scratch with improvements such as a brand new layout, more tutorials, and much more. See the new docs.
⚒️ New Features
Modelbit now automatically includes
build-essentialin all Environments that includenumpy.Added a 1-year date range option to usage charts.
Corrected the Python type hint for the
branchparameter tomb.get_dataset().
🐛 Bug Fixes
Fixed a bug where removing a user who has no API keys would still ask to confirm whether to delete those API keys.
Fixed a bug where
modelbit validatewould pass when themetadata.yamlis empty.Fixed various UX polish items in the Environment tab, Jobs tab, Logs tab, and Snowflake connection wizard.
⚒️ New Features
Increased the maximum possible output size of Redshift deployments.
Improved the performance of downloading and rendering data warehouse schemas.
Improved the error message of out-of-memory errors during
modelbit clone.Improved error messages on errors caused by missing
libgompsystem package.
🐛 Bug Fixes
Fixed incorrect syntax highlighting for Python packages in the Environment tab.
Fixed a bug that could cause the usage & billing page not to render.
Fixed a broken redirect that could occur while logging in with Okta.
Worked around a Hex change that caused us to stop parsing source code correctly in Hex notebooks.
👇 One click deployment of popular open-source ML models.
You can now deploy popular open-source models with one click. To get started, head to your workspace > deployments > click the "Add Deployment" button on the top right of the screen. Then, simply choose a model and click "Next".
⚒️ New Features
Modelbit now supports keypair auth when connecting to Snowflake.
Various quality-of-life improvements to the Snowflake connection wizard.
Cleaned up the UX of the users table in Settings.
Clarified what "Size:" means in the logs viewer.
Added copy-to-clipboard buttons to the API Endpoints screen.
🐛 Bug Fixes
Fixed a bug where copy-pasting code out of Modelbit could accidentally include spurious garbage characters.
Fixed a bug where the deeplink to an individual log line could sometimes generate a URL that 404s.
Fixed the wrapping of the deployment loading bar on tiny screens.
Fixed a memory leak in the backend service that routes inference requests.
⚒️ New Features
Improved the performance of
get_model()andset_model()on large model registries.
🐛 Bug Fixes
Fixed a bug where Modelbit did not auto-detect values of default arguments to inference functions deployed from notebooks.
Fixed a bug where Modelbit might reorder the definitions user-defined classes detected from notebooks.
Fixed a bug where adding SSH keys to Modelbit might time out.
Fixed a bug where Modelbit's terminal progress bars showed incorrect total file sizes.
Fixed a bug where Modelbit did not translate
OptionalPython type hints to SQL function definitions correctly.
⚒️ New Features
Improved error messages when users raise a Python exception that doesn't inherit from
Exception.Added logic to prevent accidental
modelbit clone'ing in directories you probably don't want to do that in, like other Git repos or~/.ssh.Added additional status messaging when
git statustakes a while because Git decides to re-filter a big file.Added retry logic and extended timeouts when adding new SSH keys as part of
modelbit clone.
🐛 Bug Fixes
Fixed an off-by-one error that misattributed training job usage that occurred in the first hour of a day.
⚒️ New Features
Improved retry logic for adding SSH keys during
modelbit clone.Added 90-day, 6-month and 9-month options to usage charts.
Made it faster to add large numbers of models to the registry at once.
Improved Python environment detection for SAM2 deployments.
Increased the number of schema entries we show in the dataset schema browser.
🐛 Bug Fixes
Fixed a bug where
get_dataset()would fail when run from a source file in a subdirectory during a training job.Fixed a bug where
modelbit.deploy()would not find variables referenced from inside withmodelbit.setup()but defined outside it.Fixed a bug where passing a Python module as data to
get_inference()would cause an infinite recursion.Fixed erroneous model summaries in the registry for models created by modules that were not installed by
pip.
⚒️ New Features
Introducing Databricks datasets! You can now use training and feature data from Databricks in your Modelbit deployments. Select "Databricks" from the "Add Warehouse" button to get started.
Made some visual improvements to the list of datasets.
Improved the error messages we show for Git sync errors.
Removed our Python package's dependency on
types-pkg-resources, which was unceremoniously disappeared from the Python supply chain last week.
🐛 Bug Fixes
Fixed a bug where you couldn't toggle dark mode during the first-time user experience.
Fixed a rare race condition during concurrent
modelbit.add_model()calls.Fixed a bug where your primary database might not show up in the schema browser if you have a lot of schema objects.
Fixed a bug where sharing and un-sharing a database with the Modelbit database user would cause all schema objects to stop showing in the schema browser.
Fixed a bug where a newly-invited user's permissions would not show up in the UI for several seconds after the invite is sent.
⚒️ New Features
Modelbit now supports Python 3.6 and Python 3.12!
Fixed a bug where very long
metadata.yamlfiles could crash deployments.Clarified the missing-dependency error message in the logs viewer.
🐛 Bug Fixes
Fixed a bug where logging very large images using
mb.log_imagecould cause pageload problems in the logs viewer.Fixed a bug where very long commit messages could scroll off the screen on the Git settings page.
Fixed a rarely-occurring Git sync bug that could cause repos to get out of sync.
⚒️ New Features
Introducing
with modelbit.setup(...), a Python API that lets you specify code to run at boot time when deploying withmodelbit.deploy()from a Notebook! Learn more.Added a Download button to the model registry UX.
Improved environment build performance!
Added an option to get Slack alerts when there are Git sync problems.
Added client-side Git validations for valid main function arguments in
metadata.yamland correctly-named top-level folders.Expanded automatic translation of PyTorch dependencies to versions with correct CUDA drivers to all versions of PyTorch.
Cleaned up some of the UX on the API Keys page.
Improved the error message when the value of the
MB_WORKSPACE_NAMEenv var is invalid or empty.
🐛 Bug Fixes
Fixed a bug where the API Keys "Last Used" field did not update when used for the management API, only for inference requests.
Fixed a bug where the Git branch settings table could overrun its container.
Fixed a bug where including
sklearninstead ofscikit-learnas a required Python package could cause your environment build to fail.Fixed a bug where Modelbit did not automatically detect dependencies used in type hints with subscripts in them.
Fixed a bug where Modelbit did not automatically detect functions that are called as Pydantic validators in Notebooks.
⚒️ New Features
Modelbit's first-time user experience now offers a Git-based deployment wizard in addition to the Notebook-based deployment wizard.
Added
modelbit deployment create(https://doc.modelbit.com/api-reference/deployment/#create) to the command-line API to make it easier to create new deployments to deploy directly with Git.Added
modelbit deployment restart(https://doc.modelbit.com/api-reference/deployment/#restart) to the command-line API for situations where you want to force a re-run of initialization code for all running copies of a deployment.modelbit clonenow automates the process of uploading an SSH key to Modelbit rather than requiring users to do it themselves before cloning.Added a download button to large model files in the source code viewer.
Logs filter settings are now saved to the URL so filtered logs views can be saved and shared!
Test calls are now green-colored in the logs filter chart for easy identification.
Added validation that only valid characters are allowed in package names in the
system_packagesargument tomodelbit.deploy(...).
🐛 Bug Fixes
Fixed a bug where Modelbit would stub large Python source files in the Git repo as if they were large binary files.
Fixed a bug where the "Last Deployed" timestamp could get truncated on the deployments index page.
Fixed a bug where, due to local caching, modifying a model in the registry could appear to modify other models in the registry if the model data structures were identical.
Revamped git deployments! When deploying via git push, Modelbit now shows you which deployments are deploying or redeploying, and links you to their deployment pages!
⚒️ New Features
Modelbit now supports files in the registry, in addition to Python objects! This is a great fit for checkpoint files for large LLMs and other neural nets. Learn more.
Added a setting to allow specific API keys to bypass branch protection.
Made many performance improvements to Modelbit's git filter for large files that runs during
modelbit cloneand some subsequent git operations.Made the time estimates in the "creating the container" stage of the new loading bar more accurate.
Improved the user experience of failed deployments on non-main branches.
Modelbit now automatically sets the
DEEPFACE_HOMEenvironment variable so that theDeepfacelibrary writes to a writeable directory in production.Improved the error message when training jobs fail because they run out of memory.
Improved the error message when users supply full URLs instead of just workspace names when logging in via API key.
Made various optimizations to the algorithm that balances running deployments across hosts.
modelbit.get_inference()is now supported inside deployments.
🐛 Bug Fixes
Fixed a bug where
modelbit.get_metrics()could be missing some metrics when called on a large number of files.Fixed a bug where the source code viewer would sometimes categorize imports and other prep code as "Modelbit-generated imports and pickled objects" when they were in fact hand-written.
Fixed a visual bug in the new deployment loading bar.
Introducing a new, more comprehensive loading graphic for deployments! This should give a more complete picture of how long a deployment will take, and which parts of the process have been completed so far.
⚒️ New Features
Added the
--branch(-b) argument to the modelbit package group of CLI commands.Made more improvements to load-balancing algorithms for GPU deployments to improve fairness and efficiency.
Made the webhooks section of the Git sync setup easier to understand.
🐛 Bug Fixes
Fixed a bug where Modelbit's Git filter would load files into memory twice, causing problems when individual model checkpoint files are larger than half the available memory.
Fixed a bug where concurrent calls to
mb.add_models()could result in some models getting dropped.Fixed a bug where
mb.add_models()could fail for large batches of small models.
⚒️ New Features
Added
mb.get_models()to efficiently fetch lots of models at once.Added
mb.set_secret()to set Modelbit secrets from notebooks.Increased the number of training jobs that can be run concurrently by a single customer.
Preserved the original types from BigQuery datasets rather than inferring them from the data.
Added retry logic to
mb.get_inference().Added a clearer error message when the word "Bearer" is unnecessarily included in the "Authorization" header in REST requests.
Improved permissions-related error messages in the BigQuery connection wizard.
Updated the styling of the "View in Modelbit" button to work in the latest versions of Hex.
🐛 Bug Fixes
Fixed a bug where training jobs could occasionally get "stuck" starting up.
Fixed a bug where models could get dropped when training jobs add thousands of models to the registry at once.
Fixed a bug where models could get dropped when multiple training jobs finish at exactly the same time.
Fixed a bug where multiple models in the registry with the exact same model data could overwrite each others' metrics.
Fixed the border color of model value descriptions in the registry in dark mode.
Fixed an incorrect performance trace label for certain types of calls to
mb.get_models().Fixed a bug where spurious quote characters could get added to training job parameter values.
Fixed a bug where training jobs would fail to run on branches with "/" characters in their names.
Fixed the occasional spurious Slack alert sent during warm-up requests.
Fixed a bug where
mb.get_branch()would return URL-formatted branch names when run inside deployments and training jobs.Fixed a bug where environments would fail to build if too many environment builds were triggered concurrently.
Fixed a bug where deployments that use thousands of models could crash the logs viewer.
Fixed a bug where deployments with no inputs would mistakenly print their logs to the "Setup" section of the logs viewer.
Fixed a bug where Modelbit would sometimes attempt to add dependencies to private packages even if those private packages have not been added to Modelbit.
Fixed a bug where
mb.models()would throw an exception instead of returning an empty list if there are no matching models.
Modelbit now supports BigQuery for datasets! Learn more.
⚒️ New Features
Modelbit now respects the order of packages specified in the
python_packagesparameter ofmodelbit.deploy().Improved the time to deploy deployments that depend on
pippackages located at customhttpsURLs.Consolidated the Mac and Linux
curlexamples as the syntax is the same. Additionally, made this the default for all users, as it appears our Windows users mostly use WSL.Cleaned up the way the Python Packages section of the Environment tab renders very long URLs.
Added an icon for
.txtfiles in the source code viewer.
🐛 Bug Fixes
Fixed a bug where very long model names could overflow the performance tracing tooltip.
Fixed a bug where you could trick Modelbit into trying to deploy with unsupported versions of Python.
Fixed a bug where Modelbit would erroneously warn that a package does not exist if it has an underscore in its name and a hyphen in its module's name.
Fixed a bug where uploading the same common file multiple times would trigger multiple redeployments.
Fixed a bug where adding the same model with the same name to the registry multiple times would cause multiple Git commits.
Fixed a bug where
.txtfiles would attempt JSON-like indentation in the source code viewer if the previously-viewed file was a JSON file.Fixed a bug where very long filenames for common files would push their filetype icons off the left-hand-side of the file viewer.
Fixed a bug where Modelbit would eagerly try to auto-detect private Git-based dependencies from notebooks and install them in production.
Modelbit now automatically issues empty "Warm-up" requests as part of automatically scaling up deployments that are under load. You can see these requests in the logs. Such deployments should see less frequent "cold start times" for real inference requests as a result.
⚒️ New Features
Added the ability to filter the logs viewer to show only requests that errored.
Modelbit is now much faster to update the latest pointer after new deployment versions have been deployed!
Improved Modelbit performance on many hundreds of simultaneous individual inference requests.
Made error messages much easier to understand when system packages are missing during environment builds.
Improved environment detection for models using OpenPrompt.
Improved the error message when an API key is invalid but API keys are not required.
Added a tip to make sure you modelbit clone to the end of the GitHub setup wizard.
🐛 Bug Fixes
Fixed the error messages when packages fail to build when adding them to private packages.
Fixed a bug where changing the name of your inference function changed it retroactively for old versions in the logs viewer.
Fixed a bug where hyphens in requirements.txt files broke the syntax highlighting.
Fixed a bug where the Snowpark error tooltip on the Overview or APIs page would sometimes not show.
Fixed a bug where very long Snowflake errors would sometimes not scroll in the Snowflake connection wizard.
Fixed a couple bugs that could cause the "Deploying" loading animation could keep showing after the deployment finished deploying.
Fixed a bug where changing the selected version after pausing auto-updates in the logs viewer would not update the logs correctly.
It was inference request performance week at Modelbit! Made various improvements to the performance of inference requests, especially the algorithms for routing incoming requests to available hosts.
⚒️ New Features
Fixed a bug where deployments with very long requirements.txt's could fail to deploy.
Fixed a bug that could very rarely cause inference requests to hang forever.
Fixed a bug where very long log lines with no spaces could run off the screen.
🐛 Bug Fixes
⚒️ New Features
Added a setting, enabled by default, to redeploy all deployments when updating a common file or private package.
For Python objects referenced from modules and unmodified since reference, Modelbit now references them from the same module in production rather than pickling them.
Made branch the first parameter to modelbit.login() as it's by far the most common.
Added client-side validation to the region parameter of modelbit.login().
Improved Python environment detection for unsloth models.
Made Python environment mismatch warnings clearer.
Made the errors clearer when modelbit.login() can't find the webserver.
Made Snowflake-side errors clearer in the Snowpark setup wizard.
🐛 Bug Fixes
Fixed a bug where the Slack alerting integration didn't work for customers on the us-east-1 cluster.
Fixed a bug where deployments could be missing from the deployments index page on non-main branches.
Fixed a bug where modal dialogs in the web app would wiggle around right before closing.
Fixed a bug where clicking the "back" button did not revert usage chart filter changes.
Fixed a bug where the Modelbit Private Cloud UX would not notice when an instance is manually shut down outside of Modelbit.
Fixed a bug where some usage charts could be broken after disabling Private Cloud.
Added a workaround for a Hex bug that rewrites Python function argspecs after cells complete, thus breaking Modelbit's Python args detection.
Announcing Date Filtering in the logs viewer! Click the "Filter" button in the logs viewer to filter down the logs. Additional improvements to the logs viewer include:
Infinite scroll to keep viewing more logs.
Additional customization of the drift charts. Use the new "Select Fields" dropdown on the drift chart to select multiple values to plot on drift charts.
⚒️ New Features
Modelbit no longer automatically retries failed environment builds. Instead, the logs from the failed build persist on the screen, and a button allows you to retry the build if you desire.
modelbit.get_dataset() now supports comparison-based filtering!
Reduced the latency of inferences on DataFrame-mode deployments by 50-70ms per call.
Keep Warm usage is now shown separately from on-demand usage so it doesn't overwhelm the on-demand usage charts and counts.
Various UX improvements to the Private Cloud setup flow.
🐛 Bug Fixes
Fixed a bug where usage charts reset their legends every 10 seconds.
Fixed a bug where model metrics were unavailable inside training jobs.
Fixed a bug where deleting a deployment might not stop its associated training jobs from running.
Fixed a bug where the deployments page could fail to update when switching between git branches.
IP Whitelisting! Head to "Settings" > "Traffic Control" to whitelist IP blocks. Also, callers' IP addresses are now included in Modelbit logs.
⚒️ New Features
Added a command-line shortcut to add all or some common files to a particular deployment. Learn more.
Modelbit now supports directory symlinks for common files in the git repo.
Modelbit no longer automatically retries failed environment builds. Instead it leaves the logs displayed in case you want to debug, with a button to retry.
Made dataset access speed about 2X faster.
By default, modelbit clone now clones from Modelbit's repo by default instead of presenting a choice.
Made rate limit errors easier to understand. Learn more.
Improved the text descriptions of large objects in their stub files in git.
Made long git sync log lines wrap more cleanly.
🐛 Bug Fixes
Fixed a bug where it was impossible to re-login inside a modelbit clone'd git repo after login expired
Fixed a bug where infrequently-called deployments could stop automatically loading new log lines in the logs viewer.
Fixed a syntax error in the example code for modelbit.get_inference.
The Modelbit logs viewer now scrolls infinitely! Get older and older log lines as you scroll. We also made some additional improvements to the logs viewer:
You can click a datapoint in the drift or runtime chart to access that datapoint’s log line.
The first-page-load performance is faster.
Very long error messages are now truncated.
Very long, very complex log lines no longer crash the browser.
⚒️ New Features
Modelbit now automatically includes local Python file dependencies in notebook-based deployments when it detects them.
Branches with only some deployments deployed now show those deployments at the top of the page.
The model registry now wraps long model names instead of truncating them.
Prevented users from accidentally requiring the google-colab Python package in deployments.
Added a warning when users try to include the ipython Python package in deployments.
🐛 Bug Fixes
Fixed a bug where using the functools.cache decorator on a function that allocates GPU RAM could cause the deployment to fail to give up the GPU when its container is terminated.
Fixed a bug where import aliases were sometimes not captured correctly from notebook-based deployments.
Fixed a bug where switching to a preexisting empty branch could cause a loading screen to show forever.
Fixed a bug that could cause spurious gaps in performance traces.
Fixed a bug where the deployment page’s three-dot menu button had a click boundary larger than the button itself.
⚒️ New Features
Added VRAM to the resource graphs in Training Jobs.
Added queue time to performance traces.
Added a "copy" button to the source code viewer.
Made various visual improvements to the registry UX, especially when comparing multiple models.
Added the modelbit.create_branch Python API method.
Updated the logs viewer to show much longer log lines in a collapsed-but-expandable fashion.
Made it clearer on the command line when you need to upgrade your modelbit python package.
🐛 Bug Fixes
Fixed a bug where creating a new branch from an individual deployment page led to a 404.
Fixed a bug where the model registry page could return a 500 error if the registry is empty.
Fixed a race condition that could cause new-deployment health checks to respond with a "Version not found" error.
When branching your git repo, Modelbit no longer automatically deploys all pre-existing deployments on the new branch. Those deployments now show a "Deploy to this branch" button on the new branch.
⚒️ New Features
Added support for custom tags in the DataDog integration. Also lower-cased our DataDog service name to modelbit.
Made improvements to the UIs for adding secrets, rate limits and SSH keys for the first time.
Added validation when passing directory names as optional parameters to modelbit clone.
Made some performance improvements to the Modelbit registry.
Added an error icon to erroring deployments on the deployment index page.
Added a loading animation to currently-deploying deployments on the deployment index page.
🐛 Bug Fixes
Fixed a bug where SSH keys could fail to add but appear to be successfully added.
Implemented the recommended workaround to a known Nvidia container crash bug.
Fixed a race condition that could cause containers to fail to attach to available GPUs depending on boot order.
Introducing billing alerts! Get notified when your usage exceeds daily, weekly or monthly thresholds.
⚒️ New Features
Modelbit no longer shows secrets and third-party integrations' API keys in the web app once they've been created.
Made various performance improvements to the Modelbit web application.
🐛 Bug Fixes
Fixed a bug where runtime tracing could crash Python 3.6 deployments.
Added some comments to Modelbit-generated code from notebook deployments to make it clearer which lines of code were generated by Modelbit.
Fixed a bug where there's a gap in performance traces for deployments that don't import modelbit.
Introducing performance tracing! Analyze your model's performance in Modelbit, including custom telemetry. Learn more.
⚒️ New Features
Added cloudpickle support to the model registry for models that don't work well with ordinary pickle! Learn more.
Added some comments to Modelbit-generated code from notebook deployments to make it clearer which lines of code were generated by Modelbit.
Made various deploy-time and cold-start-time performance improvements.
🐛 Bug Fixes
Fixed some UX bugs that can crop up when multiple git branches are deleted in quick succession.
Fixed the UI colors in the billing pop-up in dark mode.
Fixed a bug where the helpful error messages for missing packages did not trigger for models loaded from the registry.
Fixed a race condition that could cause new deployments at new workspaces to silently fail some of the time.
⚒️ New Features
Modelbit now attempts to validate the presence of required files in each deployment directory client-side before accepting git commits.
Added an integration for Anthropic. Learn more here.
🐛 Bug Fixes
Fixed a bug where very long conditionals in requirements.txt's could make the Environment page unusable.
Fixed a bug where training jobs could fail to start when created in new environments.
⚒️ New Features
Added per-deployment and system-wide health checks for connectivity to CUDA runtimes.
Added tooltips to deployment status icons.
Modelbit now sets YOLO_CONFIG_DIR to silence warnings during YOLO deployments.
Modelbit now cleans up old Snowflake inference functions when their corresponding deployments are deleted in Modelbit.
Improved request queueing logic to make better use of Modelbit's parallelism for slow deployments.
Updated the logic for auto-created workspace names to more cleanly handle ".com.region" TLDs.
Made some performance improvements to the model registry when adding many objects at a time.
🐛 Bug Fixes
Fixed a bug where very long mock Snowflake return values caused the API Endpoints screen to be unusable.
Fixed a bug where the python_packages parameter of modelbit.deploy() could fail to correctly parse environment markers.
Fixed a bug where XGBoost was sometimes not automatically detected when nested inside SKLearn objects.
Fixed a bug where hover effects didn't trigger on folders in the source code viewer in dark mode.
⚒️ New Features
Added the ability to disable retry-on-timeout logic for async inference callbacks. Learn more.
Made Modelbit's notebook output easier to read in Hex dark mode.
Made performance improvements to Modelbit's git backend server and web application.
Made various performance improvements to Snowpark deployments.
🐛 Bug Fixes
Fixed a bug where the sample code for creating Snowpark configurations was missing a use database command.
Fixed a bug where creating an account directly from a notebook login would lead to an error.
Fixed a bug where slow Snowpark deployments could error nondeterministically due to threading issues.
Modelbit now deploys directly into Snowflake compute with Snowpark Python UDFs!
⚒️ New Features
Added new Python APIs modelbit.get_snowflake_mock_return_value(...) and modelbit.set_snowflake_mock_return_value(...) to access and modify the Snowflake mock return values for already-deployed models!
Added a Python API to add metrics to an already-added model in the registry! Learn more.
Modelbit no longer automatically archives old, un-called deployments, as this was causing more problems than it was solving.
With the power of colors and text styles, made it clearer that the authentication link output by modelbit clone is in fact a link.
Added tooltips for the emojis that signify special deployment properties like GPU usage and rate limits.
Made training job error messages more helpful.
Made the login-from-notebook page much prettier.
Made some performance improvements to modelbit.deploy() from a notebook.
🐛 Bug Fixes
Fixed a bug where otherwise-successful training jobs could occasionally fail for no reason at the last minute.
Fixed an off-by-one-day bug in attributing compute minute consumption to prepaid annual plans. No customer bills were impacted.
Fixed a bug where archived deployments could re-create their Snowflake functions.
Fixed some grammar in Modelbit Python API outputs.
Fixed a bug where Snowpark inference logs time traveled into the future. Reports that Snowpark itself contains time travel functionality are unverified at this time.
⚒️ New Features
Made usage charts clearer when the "latest" version has changed over time.
Added the ability to set the return value of Snowflake mock functions. Learn more.
🐛 Bug Fixes
Fixed a bug where inference log lines with errors incorrectly reported their batch size as zero.
Fixed some incorrect syntax highlighting in private package names.
Fixed a bug where "Deploying this model" could show for deployments that only have training jobs.
Updated deployments' API Endpoints screen to include the Python API!
⚒️ New Features
Improved the performance of Modelbit datasets when used as a feature store in production.
Improved Python environment detection for YOLO models.
Added the ability to set the return value of Snowflake mock functions. Learn more.
Made additional improvements to GPU fallback logic.
🐛 Bug Fixes
Fixed a bug where Modelbit would sometimes try to pickle `None` when deploying from notebooks.
Fixed a bug where partner integrations would sometimes fail to apply themselves to running deployments.
Fixed a bug where Modelbit would sometimes 500 in scenarios with large volumes of successive batches.
Fixed a bug where Modelbit would sometimes rebuild environments unnecessarily in between training and deploying the same model.
Fixed a bug where the start time of a currently-running training job could time travel into the future.
Fixed a bug where only a subset of large Redshift batches would complete.
Modelbit now supports rate limiting! You can create custom rate limits and apply them to various deployments to throttle their execution speed.
⚒️ New Features
System metrics have been added to our status page! You can now verify Modelbit's system performance at a glance: https://status.modelbit.com/#system-metrics
In order to automatically recover models with broken internal states, Modelbit now automatically restarts models that have many consecutive failures in a short time window.
🐛 Bug Fixes
Fixed a bug where deployments whose names are prefixes of other deployments' names could confuse the git sync.
Made many improvements to GPU fallback logic and routing layer restart logic in response to: https://status.modelbit.com/incidents/xgsb85b1r8sh.
⚒️ New Features
Added support for DataFrame inputs to mb.get_inference(...) when calling DataFrame-mode deployments!
modelbit.get_inference(...) now handles chunking inputs over the wire and reassembling results so you don't have to!
modelbit.get_inference(...) now preserves the indices in your input DataFrame when calling DataFrame-mode deployments.
numpy.nan is now supported as a value in modelbit metrics (https://doc.modelbit.com/registry/metrics/)
Added an ellipsis to truncated long inputs in the logs viewer so it's clear they were truncated.
🐛 Bug Fixes
Fixed a bug where non-numerical metrics could show up in the metrics graph selector.
Fixed a bug where changing GPU settings after a failed environment build would cause the environment build logs to disappear.
Fixed a bug where a deployment's logs could disappear from the cross-version logs viewer when a new version is deployed.
Fixed a bug where mb.get_inference(...) did not work on branches with slashes in their names.
Fixed a bug where async coroutines were hidden behind a zippy in the source code viewer.
Fixed a bug that could cause the git sync to fail.
Fixed a bug where the Modelbit-generated uncomment-to-run-locally code was wrong for DataFrame-mode deployments and async deployments.
We upgraded all our runtimes this week in response to a new Linux container escape CVE. We were not impacted because we do not allow users to specify custom images, but we decided to apply the upgrade anyway out of an abundance of caution. There were no uptime impacts during the upgrade.
Model metric comparisons in the model registry! When models have the same metrics, you can visualize them with line charts and bar charts in the registry.
⚒️ New Features
Private packages are now default-excluded instead of default-included with new deployments. If their use is detected in notebooks, they are still automatically included in those notebook deployments. Learn more.
Added truncation for very long log lines in the logs viewer.
🐛 Bug Fixes
Fixed a bug where the latest version of a private package is always included in notebook deployments instead of the specified version.
Fixed issues with Modelbit Python compatibility with Python 3.7 and urllib3 < 1.26.0.
Fixed a bug where non-Python server errors triggered an unrelated Snowflake error in the Snowflake API, making these errors unknowable from Snowflake.
Fixed several bugs that led to query timeouts and errors when running single batches of hundreds of millions of inferences from Snowflake.
Fixed a bug where going to "app.modelbit.com" would throw a server error if your last-viewed git branch had a slash in its name.
Fixed a bug where linking to individual log lines would throw an error for Redshift and Snowflake log lines.
Fixed various text wrapping bugs in the web application.
Fixed a bug where the git sync status icon was sometimes overly optimistic about whether the git sync had really succeeded.
Modelbit is now SOC2 certified! To learn more, visit Modelbit's new Trust Center to download our SOC2 report and review other security measures that we maintain.
⚒️ New Features
Introducing the Python inference API! Get inferences from your deployed models right from your notebooks. Learn more!
You can now optionally include or exclude common files from individual deployments! Learn more.
You can now customize the max batch size sent from Snowflake to Modelbit for each deployment. This value gets passed directly to Snowflake as MAX_BATCH_ROWS. You can learn more here.
Improved the performance of modelbit clone on large workspaces.
🐛 Bug Fixes
Made the error message more helpful for modelbit.add_package when the package already exists yet force=True was not specified.
Fixed a couple intermittent crashes of Spark-NLP deployments.
Fixed a bug where adding more than 100 models in a single call to the registry could fail.
Model metrics! You can now store arbitrary metrics on your model. Training jobs that save models to the registry will also show you those models' metrics. Now you can always know which models had the best precision, recall, and error scores. Learn more!
⚒️ New Features
You can now link to individual training job runs! Click out the little link icon next to the output of a job run to share that output with all your coworkers.
Added support for async Python coroutines as inference functions in Modelbit.
Improved the error messages and slack alerts of datasets that fail due to missing warehouses.
Improved the readability of sklearn pipelines in model registry descriptions.
Increased the max length of the name of a git branch created from Modelbit from 25 to 100 characters.
🐛 Bug Fixes
Fixed a bug where high-usage models could make it seem like other models weren't getting any usage at all.
Fixed a bug where refreshing the model registry page and then clicking a model name led to a 404.
Integration with Tecton: Modelbit now integrates with Tecton! Any ML model and Tecton feature store can now work with your models deployed with Modelbit. Learn more.
⚒️ New Features
Common files are now implemented as symlinks in git rather than "by magic". In addition to making them more transparent, this will allow us to make several upcoming improvements to common files. Learn more.
Added a status indicator in Settings to make it clearer when there are git sync problems.
Made it clearer in individual deployments' usage charts when you're looking at usage for only one deployment version.
Inactive branches are now hidden behind a link in Git settings. This should hopefully make the page more usable for those of you with lots of git branches!
Improved Python environment detection for Spark-NLP deployments.
Modelbit now takes you to your most recently used branch when you first log in.
🐛 Bug Fixes
Fixed a bug where the OpenAI integration failed to trigger for LangChain deployments.
🎉 Happy new year from the Modelbit team!
Usage charts now show you average latency! Get model performance information at a glance.
⚒️ New Features
Modelbit now supports symlinks in deployment directories in git.
🐛 Bug Fixes
Fixed a bug where Python 3.6 deployments could fail to deploy.
Fixed a bug where usage chart export data didn't exactly match the data in the chart visualization.
Made various improvements to Modelbit's auto-scaling in bursty request scenarios.
⚒️ New Features
🐛 Bug Fixes
Fixed a bug where branches appear inactive if they have inference requests but no recent pushes.
Fixed a bug where long-running browser tabs open to very active logs viewers will just accumulate new log lines forever.
Fixed a regression in our OpenAI integration that was caused by a change in OpenAI's API.
Made it clearer on the Integrations page which integrations are active in your workspace!
⚒️ New Features
Made a number of improvements to optimize parallel inference handling.
Improved the cold start time of deployments with large disk footprints.
🐛 Bug Fixes
Fixed a bug where long inference results were not properly highlighted in the logs viewer.
Fixed a bug where inference functions in Redshift sometimes didn't update correctly.
Fixed a bug where the "last deployed" time for a model sometimes got truncated.
Fixed a bug where models sometimes didn't deploy to a non-main branch if main wasn't checked out before pushing to that branch.
Fixed a bug where the rocket badge for GPU-enabled deployments sometimes didn't show up in the version dropdown.
You can now link to individual inference log lines!
⚒️ New Features
Keep Warm is now available on non-main branches.
Made it clearer when deployments have been archived on the deployment overview page.
Improved the readability of errors in Hex notebooks.
🐛 Bug Fixes
Fixed a bug where deployments' REST API example calls didn't update very fast when "Require API Keys" was turned on.
Fixed a bug where the "Redeploy as latest" button was disabled if there's only one version.
Fixed a bug where deployments could drop pending async requests when they reboot.
Fixed a bug where mb.add_models(...) could time out when adding thousands of models.
Check out the new Model Overview page! See a useful summary of inference endpoints and training jobs in one concise view.
⚒️ New Features
The jobs logs viewer is now more useful and more visually pleasing!
You can now run multiple training jobs in a single deployment simultaneously.
Improved environment detection for Keras image models.
Enabled "redeploy as latest" even when the version is already the latest!
Lengthened the inference timeout for inference requests from Redshift.
Improved the performance of the schema browser in the dataset editor for tables with hundreds of thousands of columns.
🐛 Bug Fixes
Cleaned up a couple confusing SQL error messages in datasets.
Cleaned up some confusing warnings when deploying Keras models.
Fixed a syntax bug in the auto-generated example Redshift query.
Redesigned git settings: We made the git settings page easier to use and more visually pleasing.
⚒️ New Features
Added customizable inference timeouts with longer maximums! Learn more.
🐛 Bug Fixes
Fixed a bug where the loading animation for changes to training jobs was invisible.
Fixed a spurious warning when using the newest version of Pandas with mb.get_dataset.
Added a warning when using mb.deploy with old versions of the Modelbit package.
Fixed a bug where git sync errors sometimes did not show up in the UI.
Fixed a bug where Keras models sometimes failed to deploy from Windows environments.
Fixed a bug where Modelbit sometimes put tensorflow-intel in environments deployed from Windows instead of tensorflow.
Fixed a bug where calling Modelbit for inferences with strings-inside-strings caused errors.
A10G GPUs with 24GB VRAM have arrived at Modelbit! You can change the compute environment of any version of any deployment after it is deployed. You can also set the compute environment using git or with the require_gpu parameter of mb.deploy. Learn more.
⚒️ New Features
Overhauled the UX of the Training Jobs page to make it more visually pleasing.
Fixed various usability issues in training jobs and datasets.
🐛 Bug Fixes
Improved the layout of the Deployments page, paying special attention to displaying long deployment names!
⚒️ New Features
Upgraded all our GPUs to run CUDA 12, as required by the latest releases of PyTorch.
Improved environment detection for FastAI models.
Made Markdown files look prettier in the source code viewer.
Made it more intuitive to fix partially-completed warehouse connections.
Improved the performance of large batches of inferences.
Improved the performance of mb.get_dataset in high-concurrent-load scenarios.
🐛 Bug Fixes
GPUs are now enabled for all customers! Toggle on GPUs in the UI for inference endpoints, or select a box with a GPU for training jobs, and feel the power course through you.
⚒️ New Features
Model registry search is about 15x faster for customers with tens of thousands of models.
Deployments that aren't changed or called for a while are now automatically archived.
Archived models now correctly respect git branching and merging.
Modelbit usage charts now break out CPU and GPU usage separately.
🐛 Bug Fixes
Fixed a bug where some users got 500 errors when logging in from notebooks.
Fixed a bug where Modelbit overzealously auto-archived active deployments on new branches.
Fixed a bug where Modelbit did not correctly detect the neptune[sklearn] package.
⚒️ New Features
Made the roles assumed by API keys clearer.
Slack alerts are now available on all branches, not just main!
Owners can now restrict Users from inviting new Users.
modelbit.get_dataset() is now threadsafe.
🐛 Bug Fixes
Fixed a bug where training jobs created before their inference functions would never finish deploying.
Fixed a bug where outputs from the Python logging library did not show up in the logs viewer.
Fixed a bug where outputs to stderr did not show up in the logs viewer.
Fixed a bug where Modelbit attempted to pickle modules instead of importing them if the module was assigned to a variable.
Fixed a bug where newly-created branches would immediately show as inactive if branched from an inactive branch.
Fixed a bug where llama_cpp deployments wouldn't include necessary system dependencies if deployed using git.
Fixed a bug where we accidentally made most of our tooltips transparent.
Merge deployments from within Modelbit! Now you can merge your deployments to main (or any other branch) without tabbing over to GitHub. This means we've also turned on branch protection even when you don't have a git integration set up.
Training jobs now show CPU and Memory Usage Statistics!
⚒️ New Features
Reordered training jobs logs previews to show you the most recent logs.
Added a loading animation while training jobs are deploying.
🐛 Bug Fixes
Fixed a bug with syntax highlighting of truncated responses in the logs viewer.
Fixed a bug with spurious outputs from the add_job call when called from Hex.
Deployments Sidebar Redesign: We reorganized the Deployments sidebar into sections! If you're looking for the Notebooks section, it's been folded into the Source Code section.
⚒️ New Features
Modelbit now type-checks inputs to inference functions for Pydantic users!
Modelbit's training jobs summary now includes which deployment version was used.
Improved environment detection for Tensorflow models that use Keras.
Improved the performance of the Usage Data graphs.
🐛 Bug Fixes
Fixed a bug where if you use a custom PyTorch version locally, we always incorporated that version in production, even if you explicitly specified a different version in python_packages.
Fixed various bugs related to non-Anaconda Windows environments.
Fixed various search quality issues in the model registry search.
Fixed usability issues relating to long model names and long source code file names.
Slack Alerts: You can now get Slack alerts when datasets fail to update and when training jobs fail. Learn more.
⚒️ New Features
Image files now render previews in the source code viewer.
Added a loading bar when adding lots of models to the registry at once.
Improved the performance of large GPU training jobs.
Modelbit's logs viewer is faster.
Improved environment detection for pycaret models.
Improved the performance of models using PyTorch 2.1.0.
Greatly improved the error messages from mb.get_model().
Improved performance and memory usage when deploying models that contain extra_files.
🐛 Bug Fixes
Fixed a bug where mb.models() did not work on branches with slashes in their names.
Fixed a bug with syntax highlighting of truncated responses in the logs viewer.
Fixed a bug where imports inside of functions were not detected as dependencies.
Llama 2: You can now deploy Llama 2 Models in Modelbit with vastly improved support for llama.cpp. Learn more.
⚒️ New Features
Improved the freshness and latency of Modelbit usage data.
Added a helpful human-readable tooltip to the jobs cron scheduler.
Made it easier to find the checkbox to require API keys.
🐛 Bug Fixes
Fixed a bug where deployments would claim to be deployed sooner than they were.
Fixed a bug where GPU training jobs sometimes did not report their usage.
Fixed various UX issues caused by very long jobs log outputs.
Fixed a bug where Modelbit's first-time user experience didn't work correctly in interactive Python terminals.
Fixed a bug where Modelbit wouldn't connect to fresh git repos because they contain .git files.
Training ML Models on GPUs: We've added support for training models on GPUs and on much larger boxes! Boxes even larger than the ones chooseable in the product are available upon request. Learn more.
Large Python Environments: We made it faster to build large Python environments as part of deploying large models! For example, the time to build a 6GB environment for OWL-ViT on JAX has been reduced from ~280s to ~180s. We are continuing to work on making it faster to build environments and deploy models.
⚒️ New Features
We revamped the git connection wizard to make it easier to connect your own git repo.
Modelbit is now available in AWS us-east-1! This is great news for Enterprise customers who want to leverage AWS PrivateLink in that region.
Added support for Azure DevOps Git Repos. Learn more.
We now support vertical pipes ("|") in model names in case you make questionable naming decisions.
The mb.models() API call is now supported inside of deployments. Learn more.
You can now add a secret header to Modelbit logs webhooks for security purposes. Learn more.
🐛 Bug Fixes
Fixed a bug where training jobs that save changes on success would fail on protected branches.
Fixed a bug where dataset refresh histories were displayed in UTC instead of your local timezone.
Fixed a bug where models in the model registry sometimes did not branch correctly.
Fixed a bug where scheduled training jobs sometimes appeared to time travel into the past when running.