[ { "query_id": "5e878c76-25c1-4bad-8cae-6a40ca4c8138", "query": "experiment tracking", "matching_document": "## Track Experiments\n### How it works\nTrack a machine learning experiment with a few lines of code:\n1. Create a W&B run.\n2. Store a dictionary of hyperparameters, such as learning rate or model type, into your configuration (`wandb.config`).\n3. Log metrics (`wandb.log()`) over time in a training loop, such as accuracy and loss.\n4. Save outputs of a run, like the model weights or a table of predictions. \n\nThe proceeding pseudocode demonstrates a common W&B Experiment tracking workflow: \n\n```python showLineNumbers\n\n# 1. Start a W&B Run\n\nwandb.init(entity=\"\", project=\"my-project-name\")\n\n# 2. Save mode inputs and hyperparameters\n\nwandb.config.learning\\_rate = 0.01\n\n# Import model and data\n\nmodel, dataloader = get\\_model(), get\\_data()\n\n# Model training code goes here\n\n# 3. Log metrics over time to visualize performance\n\nwandb.log({\"loss\": loss})\n\n# 4. Log an artifact to W&B\n\nwandb.log\\_artifact(model)\n```", "matching_document_document_id": "1c7f8798-7b2a-4baa-9829-14ada61db6bc", "query_weight": 0.1 }, { "query_id": "d7b77e8a-e86c-4953-bc9f-672618cdb751", "query": "Bayesian optimization", "matching_document": "## Methods for Automated Hyperparameter Optimization\n### Bayesian Optimization\nBayesian optimization is a hyperparameter tuning technique that uses a surrogate function to determine the next set of hyperparameters to evaluate. In contrast to grid search and random search, Bayesian optimization is an informed search method. \n\n### Inputs \n\n* A set of hyperparameters you want to optimize\n* A continuous search space for each hyperparameter as a value range\n* A performance metric to optimize\n* Explicit number of runs: Because the search space is continuous, you must manually stop the search or define a maximum number of runs. \n\nThe differences in grid search are highlighted in bold above. \n\nA popular way to implement Bayesian optimization in Python is to use BayesianOptimization from the [bayes_opt](https://github.com/fmfn/BayesianOptimization) library. Alternatively, as shown below, you can set up Bayesian optimization for hyperparameter tuning with W&B. \n\n### Steps \n\n### Output \n\n### Advantages \n\n### Disadvantages", "matching_document_document_id": "d11e5989-da08-4543-b54c-cbce0edd4c82", "query_weight": 0.7432258064516128 }, { "query_id": "023fca2d-78cb-4cc1-ad11-1e054428a891", "query": "How to integrate Weights & Biases with PyTorch?", "matching_document": "## \ud83d\udd25 = W&B \u2795 PyTorch\n\nUse Weights & Biases for machine learning experiment tracking, dataset versioning, and project collaboration. \n\n## What this notebook covers: \n\nWe show you how to integrate Weights & Biases with your PyTorch code to add experiment tracking to your pipeline. \n\n## The resulting interactive W&B dashboard will look like: \n\n## In pseudocode, what we'll do is: \n\n```\n# import the library\nimport wandb\n\n# start a new experiment\nwandb.init(project=\"new-sota-model\")\n\n#\u2003capture a dictionary of hyperparameters with config\nwandb.config = {\"learning\\_rate\": 0.001, \"epochs\": 100, \"batch\\_size\": 128}\n\n# set up model and data\nmodel, dataloader = get\\_model(), get\\_data()\n\n# optional: track gradients\nwandb.watch(model)\n\nfor batch in dataloader:\nmetrics = model.training\\_step()\n#\u2003log metrics inside your training loop to visualize model performance\nwandb.log(metrics)\n\n# optional: save model at the end\nmodel.to\\_onnx()\nwandb.save(\"model.onnx\")\n\n``` \n\n## Follow along with a video tutorial!", "matching_document_document_id": "ee62983f-f238-4b4a-bde8-6b4c068d7918", "query_weight": 1.2541935483870967 }, { "query_id": "188162c6-744e-4bd1-b7b7-9146fcb00068", "query": "model registry W&B", "matching_document": "## Register models\n### Model Registry\nAfter logging a bunch of checkpoints across multiple runs during experimentation, now comes time to hand-off the best checkpoint to the next stage of the workflow (e.g. testing, deployment). \n\nThe Model Registry is a central page that lives above individual W&B projects. It houses **Registered Models**, portfolios that store \"links\" to the valuable checkpoints living in individual W&B Projects. \n\nThe model registry offers a centralized place to house the best checkpoints for all your model tasks. Any `model` artifact you log can be \"linked\" to a Registered Model. \n\n### Creating **Registered Models** and Linking through the UI \n\n#### 1. Access your team's model registry by going the team page and selecting `Model Registry` \n\n#### 2. Create a new Registered Model. \n\n#### 3. Go to the artifacts tab of the project that holds all your model checkpoints \n\n#### 4. Click \"Link to Registry\" for the model artifact version you want. \n\n### Creating Registered Models and Linking through the **API**", "matching_document_document_id": "4585f985-3076-45c3-8026-7e41748068f1", "query_weight": 0.1 }, { "query_id": "cb7b845c-ada3-4a58-a9fa-5b0dc875967a", "query": "What is Weights & Biases?", "matching_document": "## What is W&B?\n\nWeights & Biases (W&B) is the AI developer platform, with tools for training models, fine-tuning models, and leveraging foundation models. \n\nSet up W&B in 5 minutes, then quickly iterate on your machine learning pipeline with the confidence that your models and data are tracked and versioned in a reliable system of record. \n\nThis diagram outlines the relationship between W&B products. \n\n**W&B Models** is a set of lightweight, interoperable tools for machine learning practitioners training and fine-tuning models.\n- Experiments: Machine learning experiment tracking\n- Model Registry: Manage production models centrally\n- Launch: Scale and automate workloads\n- Sweeps: Hyperparameter tuning and model optimization \n\n**W&B Prompts** is for LLM debugging and monitoring, including usage of OpenAI's GPT API.", "matching_document_document_id": "7178c033-7049-4ab8-b4bb-2d7b71a495a8", "query_weight": 0.7432258064516128 }, { "query_id": "4bd23890-f2f4-4965-8e36-4c61e1a806cc", "query": "Weights & Biases integration OpenAI", "matching_document": "## OpenAI API\n\nUse the Weights & Biases OpenAI API integration to log requests, responses, token counts and model metadata with 1 line of code for all OpenAI models, including fine-tuned models. \n\n:::info\nThe W&B autolog integration works with `openai <= 0.28.1`. Install the correct version of `openai` with `pip install openai==0.28.1`.\n::: \n\n**Try in a Colab Notebook here \u2192** \n\nWith just 1 line of code you can now automatically log inputs and outputs from the OpenAI Python SDK to Weights & Biases! \n\nOnce you start logging your API inputs and outputs you can quickly evaluate the performance of difference prompts, compare different model settings (such as temperature), and track other usage metrics such as token usage. \n\nTo get started, pip install the `wandb` library, then follow the steps below: \n\n### 1. Import autolog and initialise it \n\nFirst, import `autolog` from `wandb.integration.openai` and initialise it. \n\n### 2. Call the OpenAI API \n\n### 3. View your OpenAI API inputs and responses", "matching_document_document_id": "04695bcc-0c96-4e1e-baf7-113b1b4dca09", "query_weight": 0.9754838709677419 }, { "query_id": "cbbf4fe7-c053-4d54-bc84-e3bf2d6e9f7f", "query": "PyTorch Weights & Biases integration tutorial", "matching_document": "## \ud83d\udd25 = W&B \u2795 PyTorch\n\nUse Weights & Biases for machine learning experiment tracking, dataset versioning, and project collaboration. \n\n## What this notebook covers: \n\nWe show you how to integrate Weights & Biases with your PyTorch code to add experiment tracking to your pipeline. \n\n## The resulting interactive W&B dashboard will look like: \n\n## In pseudocode, what we'll do is: \n\n```\n# import the library\nimport wandb\n\n# start a new experiment\nwandb.init(project=\"new-sota-model\")\n\n#\u2003capture a dictionary of hyperparameters with config\nwandb.config = {\"learning\\_rate\": 0.001, \"epochs\": 100, \"batch\\_size\": 128}\n\n# set up model and data\nmodel, dataloader = get\\_model(), get\\_data()\n\n# optional: track gradients\nwandb.watch(model)\n\nfor batch in dataloader:\nmetrics = model.training\\_step()\n#\u2003log metrics inside your training loop to visualize model performance\nwandb.log(metrics)\n\n# optional: save model at the end\nmodel.to\\_onnx()\nwandb.save(\"model.onnx\")\n\n``` \n\n## Follow along with a video tutorial!", "matching_document_document_id": "ee62983f-f238-4b4a-bde8-6b4c068d7918", "query_weight": 1.2541935483870967 }, { "query_id": "cf7112a4-5fe1-459d-ba6b-42ef8bb4a2d4", "query": "Best practices for managing multiple runs in WANDB.", "matching_document": "## \u270d\ufe0f W&B Best Practices\n\n1. **Projects**: Log multiple runs to a project to compare them. `wandb.init(project=\"project-name\")`\n2. **Groups**: For multiple processes or cross validation folds, log each process as a runs and group them together. `wandb.init(group=\"experiment-1\")`\n3. **Tags**: Add tags to track your current baseline or production model.\n4. **Notes**: Type notes in the table to track the changes between runs.\n5. **Reports**: Take quick notes on progress to share with colleagues and make dashboards and snapshots of your ML projects.", "matching_document_document_id": "aa2dbfa2-471e-4ec7-b1a2-60a608aa6873", "query_weight": 0.8361290322580645 }, { "query_id": "7eff21b3-8ec2-4279-b3a6-e4923aa5ec87", "query": "How to integrate YOLOv5 with Weights & Biases?", "matching_document": "## \ud83d\udce6 YOLOv5 +\ud83e\ude84 W&B\n\nTo install YOLOv5 with W&B integration all we need to is simply [clone this repo](https://github.com/awsaf49/yolov5-wandb). This s repo is synced with official YOLOv5 repo so you'll get the latest features of YOLOv5 along with support of W&B. \n\n> \n\n```\n!git clone https://github.com/awsaf49/yolov5-wandb.git yolov5 # clone\n%cd yolov5\n%pip install -qr requirements.txt # install\n\nimport torch\nimport utils\ndisplay = utils.notebook_init() # checks\n```", "matching_document_document_id": "7d15ab04-df03-4e91-bf66-37d06f5577e3", "query_weight": 1.2541935483870967 }, { "query_id": "839e408e-dfe8-4256-b4b8-3993b0588554", "query": "tracking experiments with wandb", "matching_document": "## Track Experiments\n### How it works\nTrack a machine learning experiment with a few lines of code:\n1. Create a W&B run.\n2. Store a dictionary of hyperparameters, such as learning rate or model type, into your configuration (`wandb.config`).\n3. Log metrics (`wandb.log()`) over time in a training loop, such as accuracy and loss.\n4. Save outputs of a run, like the model weights or a table of predictions. \n\nThe proceeding pseudocode demonstrates a common W&B Experiment tracking workflow: \n\n```python showLineNumbers\n\n# 1. Start a W&B Run\n\nwandb.init(entity=\"\", project=\"my-project-name\")\n\n# 2. Save mode inputs and hyperparameters\n\nwandb.config.learning\\_rate = 0.01\n\n# Import model and data\n\nmodel, dataloader = get\\_model(), get\\_data()\n\n# Model training code goes here\n\n# 3. Log metrics over time to visualize performance\n\nwandb.log({\"loss\": loss})\n\n# 4. Log an artifact to W&B\n\nwandb.log\\_artifact(model)\n```", "matching_document_document_id": "1c7f8798-7b2a-4baa-9829-14ada61db6bc", "query_weight": 0.1 }, { "query_id": "195b8ea5-165c-4555-baea-4e8e37e2001e", "query": "Custom chart creation in Weights & Biases using Vega", "matching_document": "## Step 2: Create a custom chart for the confusion matrix\nW&B custom charts are written in [Vega](https://vega.github.io/vega/), a powerful and flexible visualization language. You can find many examples and walkthroughs online, and it can help to start with an existing preset that is most similar to your desired custom visualization. You can iterate from small changes in our [IDE](https://wandb.ai/wandb/posts/reports/The-W-B-Machine-Learning-Visualization-IDE--VmlldzoyOTQxNDY5/edit), which renders the plot as you change its definition. \n\nHere is the full Vega spec for this multi-class confusion matrix: \n\n* From your project workspace or report, click on \"Add a visualization\" and select \"Custom chart\"\n* Pick any existing preset and replace its definition with the Vega spec below\n* Click \"Save As\" to give this preset a name for easier reference (I recommend \"confusion_matrix\" :)", "matching_document_document_id": "0c28a653-58b5-42dc-9742-7cb178bdee9b", "query_weight": 1.8116129032258064 }, { "query_id": "ec601db6-c291-49fd-bb92-fdace481e9f7", "query": "Custom charts in Wandb", "matching_document": "## Custom Charts\n\nUse **Custom Charts** to create charts that aren't possible right now in the default UI. Log arbitrary tables of data and visualize them exactly how you want. Control details of fonts, colors, and tooltips with the power of Vega. \n\n* **What's possible**: Read the launch announcement \u2192\n* **Code**: Try a live example in a hosted notebook \u2192\n* **Video**: Watch a quick walkthrough video \u2192\n* **Example**: Quick Keras and Sklearn demo notebook \u2192 \n\n### How it works \n\n1. **Log data**: From your script, log config and summary data as you normally would when running with W&B. To visualize a list of multiple values logged at one specific time, use a custom`wandb.Table`\n2. **Customize the chart**: Pull in any of this logged data with a GraphQL query. Visualize the results of your query with Vega, a powerful visualization grammar.\n3. **Log the chart**: Call your own preset from your script with `wandb.plot_table()`. \n\n## Log charts from a script \n\n### Builtin presets \n\n### Custom presets", "matching_document_document_id": "b2c63377-14e4-44f2-8002-0a7731227e09", "query_weight": 0.8361290322580645 }, { "query_id": "f77585d7-4ea4-4269-85f9-e83f64c6bdde", "query": "WandB logging best practices", "matching_document": "## \u270d\ufe0f W&B Best Practices\n\n1. **Projects**: Log multiple runs to a project to compare them. `wandb.init(project=\"project-name\")`\n2. **Groups**: For multiple processes or cross validation folds, log each process as a runs and group them together. `wandb.init(group=\"experiment-1\")`\n3. **Tags**: Add tags to track your current baseline or production model.\n4. **Notes**: Type notes in the table to track the changes between runs.\n5. **Reports**: Take quick notes on progress to share with colleagues and make dashboards and snapshots of your ML projects.", "matching_document_document_id": "aa2dbfa2-471e-4ec7-b1a2-60a608aa6873", "query_weight": 0.9754838709677419 }, { "query_id": "6788e56f-588b-4707-998e-dd8f129b3bc7", "query": "updating artifacts in W&B", "matching_document": "## Update artifacts\n```\nimport wandb\n\napi = wandb.Api()\n\nartifact = api.artifact(\"entity/project/artifact:alias\")\n\n# Update the description\nartifact.description = \"My new description\"\n\n# Selectively update metadata keys\nartifact.metadata[\"oldKey\"] = \"new value\"\n\n# Replace the metadata entirely\nartifact.metadata = {\"newKey\": \"new value\"}\n\n# Add an alias\nartifact.aliases.append(\"best\")\n\n# Remove an alias\nartifact.aliases.remove(\"latest\")\n\n# Completely replace the aliases\nartifact.aliases = [\"replaced\"]\n\n# Persist all artifact modifications\nartifact.save()\n\n``` \n\nFor more information, see the Weights and Biases Artifact API.", "matching_document_document_id": "526f6e37-2a3e-4594-8b0c-f79dc0178288", "query_weight": 1.3935483870967742 }, { "query_id": "ed67c0e9-84cf-423e-9eb6-a9bc80abb8c9", "query": "PyTorch integration with Weights & Biases", "matching_document": "## \ud83d\udd25 = W&B \u2795 PyTorch\n\nUse Weights & Biases for machine learning experiment tracking, dataset versioning, and project collaboration. \n\n## What this notebook covers: \n\nWe show you how to integrate Weights & Biases with your PyTorch code to add experiment tracking to your pipeline. \n\n## The resulting interactive W&B dashboard will look like: \n\n## In pseudocode, what we'll do is: \n\n```\n# import the library\nimport wandb\n\n# start a new experiment\nwandb.init(project=\"new-sota-model\")\n\n#\u2003capture a dictionary of hyperparameters with config\nwandb.config = {\"learning\\_rate\": 0.001, \"epochs\": 100, \"batch\\_size\": 128}\n\n# set up model and data\nmodel, dataloader = get\\_model(), get\\_data()\n\n# optional: track gradients\nwandb.watch(model)\n\nfor batch in dataloader:\nmetrics = model.training\\_step()\n#\u2003log metrics inside your training loop to visualize model performance\nwandb.log(metrics)\n\n# optional: save model at the end\nmodel.to\\_onnx()\nwandb.save(\"model.onnx\")\n\n``` \n\n## Follow along with a video tutorial!", "matching_document_document_id": "ee62983f-f238-4b4a-bde8-6b4c068d7918", "query_weight": 1.2541935483870967 }, { "query_id": "04dbadfc-e290-4e9e-af5d-ab334feb75f3", "query": "Weights & Biases sweeps", "matching_document": "## Sweep 101\n\nUse Weights & Biases Sweeps to automate hyperparameter optimization and explore the space of possible models. \n\n## Check out Hyperparameter Optimization in PyTorch using W&B Sweeps $\\rightarrow$ \n\nRunning a hyperparameter sweep with Weights & Biases is very easy. There are just 3 simple steps: \n\n1. **Define the sweep:** We do this by creating a dictionary or a YAML file that specifies the parameters to search through, the search strategy, the optimization metric et all.\n2. **Initialize the sweep:**\n`sweep_id = wandb.sweep(sweep_config)`\n3. **Run the sweep agent:**\n`wandb.agent(sweep_id, function=train)` \n\nAnd voila! That's all there is to running a hyperparameter sweep! In the notebook below, we'll walk through these 3 steps in more detail.", "matching_document_document_id": "81a7053e-af10-4e08-842f-9318a01088ff", "query_weight": 0.5574193548387096 }, { "query_id": "a73fa825-a2ff-4dee-8e51-2adb9c0cefb7", "query": "hyperparameter optimization", "matching_document": "## How Do You Optimize Hyperparameters?\nBefore starting with automated hyperparameter optimization, you need to specify all of the above. But you can adjust the hyperparameter search space and the number of trial runs during manual hyperparameter tuning. \n\nThe\u00a0generic steps\u00a0for hyperparameter tuning are: \n\n* Select a set of hyperparameter values to evaluate\n* Run an ML experiment for the selected set of hyperparameters and their values, and evaluate and log its performance metric.\n* Repeat for the specified number of trial runs or until you are happy with the model\u2019s performance \n\nDepending on whether you manually conduct these steps or automate them, we talk about\u00a0manual or automated hyperparameter optimization. \n\nAfter\u00a0this process, you will end up with a list of experiments, including their hyperparameters and performance metrics. An automated hyperparameter optimization algorithm returns the experiment with the best performance metric and the respective hyperparameter values.", "matching_document_document_id": "2263b726-f63d-49a2-a603-bd48c4e5a70b", "query_weight": 0.8825806451612903 }, { "query_id": "bdb107ca-cbe1-43b0-b235-ffb516205185", "query": "wandb.init() code saving", "matching_document": "--- \n\n## displayed\\_sidebar: default \n\n# Code Saving \n\nBy default, we only save the latest git commit hash. You can turn on more code features to compare the code between your experiments dynamically in the UI. \n\nStarting with `wandb` version 0.8.28, we can save the code from your main training file where you call `wandb.init()`. This will get sync'd to the dashboard and show up in a tab on the run page, as well as the Code Comparer panel. Go to your settings page to enable code saving by default. \n\n## Save Library Code \n\nWhen code saving is enabled, wandb will save the code from the file that called `wandb.init()`. To save additional library code, you have two options: \n\n* Call `wandb.run.log_code(\".\")` after calling `wandb.init()`\n* Pass a settings object to `wandb.init` with code\\_dir set: `wandb.init(settings=wandb.Settings(code_dir=\".\"))` \n\n## Code Comparer \n\n## Jupyter Session History \n\n## Jupyter diffing", "matching_document_document_id": "10c3ed4d-db30-4f77-b9c9-d90af6894203", "query_weight": 1.1148387096774193 }, { "query_id": "fe3519f2-3d05-43a1-b52b-5fd806bdfd49", "query": "W&B artifacts", "matching_document": "## Register models\n### Log Data and Model Checkpoints as Artifacts\nW&B Artifacts allows you to track and version arbitrary serialized data (e.g. datasets, model checkpoints, evaluation results). When you create an artifact, you give it a name and a type, and that artifact is forever linked to the experimental system of record. If the underlying data changes, and you log that data asset again, W&B will automatically create new versions through checksummming its contents. W&B Artifacts can be thought of as a lightweight abstraction layer on top of shared unstructured file systems. \n\n### Anatomy of an artifact \n\nThe `Artifact` class will correspond to an entry in the W&B Artifact registry. The artifact has\n\\* a name\n\\* a type\n\\* metadata\n\\* description\n\\* files, directory of files, or references \n\nExample usage: \n\n```\nrun = wandb.init(project=\"my-project\")\nartifact = wandb.Artifact(name=\"my\\_artifact\", type=\"data\")\nartifact.add\\_file(\"/path/to/my/file.txt\")\nrun.log\\_artifact(artifact)\nrun.finish()\n\n```", "matching_document_document_id": "686be7b2-df58-4b71-ab38-467a2ac4add0", "query_weight": 1.3935483870967742 }, { "query_id": "448056d9-0e40-4723-8c39-20c50e98aa74", "query": "Wandb custom charts", "matching_document": "## Custom Charts Walkthrough\n\nTo go beyond the built-in charts in W&B, use the new **Custom Charts** feature to control the details of exactly what data you're loading in to a panel and how you visualize that data. \n\n**Overview** \n\n1. Log data to W&B\n2. Create a query\n3. Customize the chart \n\n## 1. Log data to W&B \n\nFirst, log data in your script. Use wandb.config for single points set at the beginning of training, like hyperparameters. Use wandb.log() for multiple points over time, and log custom 2D arrays with wandb.Table(). We recommend logging up to 10,000 data points per logged key. \n\n```\n# Logging a custom table of data\nmy\\_custom\\_data = [[x1, y1, z1], [x2, y2, z2]]\nwandb.log(\n{\"custom\\_data\\_table\": wandb.Table(data=my\\_custom\\_data, columns=[\"x\", \"y\", \"z\"])}\n)\n\n``` \n\nTry a quick example notebook to log the data tables, and in the next step we'll set up custom charts. See what the resulting charts look like in the live report. \n\n## 2. Create a query \n\n## 3. Customize the chart", "matching_document_document_id": "9e5387e4-cb3f-4536-a1ea-23be13cf77fb", "query_weight": 0.8361290322580645 }, { "query_id": "8cebd252-add6-4c93-80f2-a729d433ce4a", "query": "W&B best practices", "matching_document": "## \u270d\ufe0f W&B Best Practices\n\n1. **Projects**: Log multiple runs to a project to compare them. `wandb.init(project=\"project-name\")`\n2. **Groups**: For multiple processes or cross validation folds, log each process as a runs and group them together. `wandb.init(group=\"experiment-1\")`\n3. **Tags**: Add tags to track your current baseline or production model.\n4. **Notes**: Type notes in the table to track the changes between runs.\n5. **Reports**: Take quick notes on progress to share with colleagues and make dashboards and snapshots of your ML projects.", "matching_document_document_id": "aa2dbfa2-471e-4ec7-b1a2-60a608aa6873", "query_weight": 1.3470967741935482 }, { "query_id": "bd68547f-07ab-4b4f-a7b0-95731f8a282c", "query": "Are there any best practices for using wandb in a distributed training environment?", "matching_document": "## Add wandb to Any Library\n#### Distributed Training\n\nFor frameworks supporting distributed environments, you can adapt any of the following workflows: \n\n* Detect which is the \u201cmain\u201d process and only use `wandb` there. Any required data coming from other processes must be routed to the main process first. (This workflow is encouraged).\n* Call `wandb` in every process and auto-group them by giving them all the same unique `group` name \n\nSee Log Distributed Training Experiments for more details", "matching_document_document_id": "c4a1f7d0-82d6-411a-890c-38d5ac3e6e48", "query_weight": 0.7432258064516128 }, { "query_id": "d80de3d8-06b7-45d3-9447-a6c847b6315f", "query": "How to invite team members to a project in Weights & Biases?", "matching_document": "## Teams\n### Invite team members\nInvite new members to your team.\n1. Ensure the team member already has a W&B Account.\n2. Navigate to .\n3. Select **Manage members**.\n4. A model will appear. Provide the username or email for the **Email or Username** field, select a team for them to join from the **Team** dropdown menu, and select a role type from the **Organizational Role** dropdown menu. \n\n1. Select the **Add** button. \n\n:::info\n\\* If you have an Enterprise account, please contact your Account Executive to invite new members to your team.\n:::", "matching_document_document_id": "5c1c7d28-53bf-45fb-89c2-1219c423b66f", "query_weight": 2.136774193548387 }, { "query_id": "d80c7b46-59c3-4e6f-b377-ee4276e22397", "query": "wandb.init best practices", "matching_document": "## \u270d\ufe0f W&B Best Practices\n\n1. **Projects**: Log multiple runs to a project to compare them. `wandb.init(project=\"project-name\")`\n2. **Groups**: For multiple processes or cross validation folds, log each process as a runs and group them together. `wandb.init(group=\"experiment-1\")`\n3. **Tags**: Add tags to track your current baseline or production model.\n4. **Notes**: Type notes in the table to track the changes between runs.\n5. **Reports**: Take quick notes on progress to share with colleagues and make dashboards and snapshots of your ML projects.", "matching_document_document_id": "aa2dbfa2-471e-4ec7-b1a2-60a608aa6873", "query_weight": 1.1148387096774193 }, { "query_id": "dfff1f01-8dd1-46bc-84bf-78783878d810", "query": "building LLM-powered apps with W&B", "matching_document": "## Prompts for LLMs\n\nW&B Prompts is a suite of LLMOps tools built for the development of LLM-powered applications. Use W&B Prompts to visualize and inspect the execution flow of your LLMs, analyze the inputs and outputs of your LLMs, view the intermediate results and securely store and manage your prompts and LLM chain configurations. \n\n## Use Cases \n\nW&B Prompts provides several solutions for building and monitoring LLM-based apps. Software developers, prompt engineers, ML practitioners, data scientists, and other stakeholders working with LLMs need cutting-edge tools to: \n\n* Explore and debug LLM chains\u00a0and prompts with greater granularity.\n* Monitor and observe LLMs to better understand and evaluate performance, usage, and budgets. \n\n## Products \n\n### Traces \n\nW&B\u2019s LLM tool is called\u00a0*Traces*.\u00a0**Traces**\u00a0allow you to track and visualize the inputs and outputs, execution flow, model architecture, and any intermediate results of your LLM chains. \n\n### Weave \n\n### How it works \n\n## Integrations", "matching_document_document_id": "99fd9bc0-a4ff-4291-9e1c-c6aab2b9b31a", "query_weight": 0.1 }, { "query_id": "2bf96622-ec30-45e9-a83c-fee84e3e685c", "query": "Weights & Biases features", "matching_document": "## Weights & Biases overview\nW&B is a platform that helps data scientists track their models, datasets, system information and more. With a few lines of code, you can start tracking everything about these features. It's free for personal use. Team use is normally a paid utility, but teams for academic purposes are free. You can use W&B with your favourite framework, like TensorFlow, Keras, PyTorch, Sklearn, fastai and many others. \n\nAll tracking information is sent to a dedicated project page on the W&B UI, where you can open high quality visualizations, aggregate information and compare models or parameters. One of the advantages of remotely storing the experiment\u2019s information is that it is easy to collaborate on the same project and share the results with your teammates. \n\nW&B provides 4 useful tools: \n\n* Dashboard: Experiment tracking\n* Artifacts: Dataset versioning, model versioning\n* Sweeps: Hyperparameter optimization\n* Reports: Save and share reproducible findings", "matching_document_document_id": "8c1600ad-9b6e-45cd-86ae-03afd8f80332", "query_weight": 0.9754838709677419 }, { "query_id": "ecdc6050-d98d-481c-8770-cc4a083e8aa3", "query": "Weights & Biases artifacts", "matching_document": "## Artifacts\n### How it works\nCreate an artifact with four lines of code:\n1. Create a W&B run.\n2. Create an artifact object with the `wandb.Artifact` API.\n3. Add one or more files, such as a model file or dataset, to your artifact object.\n4. Log your artifact to W&B. \n\n`python showLineNumbers\nrun = wandb.init(project=\"artifacts-example\", job_type=\"add-dataset\")\nartifact = wandb.Artifact(name=\"my_data\", type=\"dataset\")\nartifact.add_dir(local_path=\"./dataset.h5\") # Add dataset directory to artifact\nrun.log_artifact(artifact) # Logs the artifact version \"my_data:v0\"` \n\n:::tip\nThe preceding code snippet, and the colab linked on this page, show how to track files by uploading them to W&B. See the track external files page for information on how to add references to files or directories that are stored in external object storage (for example, in an Amazon S3 bucket).\n:::", "matching_document_document_id": "01d94aae-eec0-46ba-9972-915e551f3d88", "query_weight": 0.5109677419354839 }, { "query_id": "ea5d6313-2e41-49ef-b455-0f3fd335d513", "query": "wandb confusion matrix chart", "matching_document": "## Step 2: Create a custom chart for the confusion matrix\n```\n{\n\"$schema\": \"https://vega.github.io/schema/vega-lite/v4.json\",\n\"description\": \"Multi-class confusion matrix\",\n\"data\": {\n\"name\": \"wandb\"\n},\n\"width\": 40,\n\"height\": {\"step\":6},\n\"spacing\": 5,\n\"mark\" : \"bar\",\n\"encoding\": {\n\"y\": {\"field\": \"name\", \"type\": \"nominal\", \"axis\" : {\"labels\" : false},\n\"title\" : null, \"scale\": {\"zero\": false}},\n\"x\": {\n\"field\": \"${field:count}\",\n\"type\": \"quantitative\",\n\"axis\" : null,\n\"title\" : null\n},\n\"tooltip\": [\n{\"field\": \"${field:count}\", \"type\": \"quantitative\", \"title\" : \"Count\"},\n{\"field\": \"name\", \"type\": \"nominal\", \"title\" : \"Run name\"}\n],\n\"color\": {\n\"field\": \"name\",\n\"type\": \"nominal\",\n\"legend\": {\"orient\": \"top\", \"titleOrient\": \"left\"},\n\"title\": \"Run name\"\n},\n\"row\": {\"field\": \"${field:actual}\", \"title\": \"Actual\",\n\"header\": {\"labelAlign\" : \"left\", \"labelAngle\": 0}},\n\"column\": {\"field\": \"${field:predicted}\", \"title\": \"Predicted\"}\n}\n}\n```", "matching_document_document_id": "c2a16267-147d-4bc0-9ecc-390cf7b0fca2", "query_weight": 0.8361290322580645 }, { "query_id": "4ee9a123-0dee-447d-9033-d792b28d8382", "query": "How does wandb.save function and what are its use cases?", "matching_document": "## Save your machine learning model\n* Use wandb.save(filename).\n* Put a file in the wandb run directory, and it will get uploaded at the end of the run. \n\nIf you want to sync files as they're being written, you can specify a filename or glob in wandb.save. \n\nHere's how you can do this in just a few lines of code. See [this colab](https://colab.research.google.com/drive/1pVlV6Ua4C695jVbLoG-wtc50wZ9OOjnC) for a complete example. \n\n```\n# \"model.h5\" is saved in wandb.run.dir & will be uploaded at the end of training\nmodel.save(os.path.join(wandb.run.dir, \"model.h5\"))\n\n# Save a model file manually from the current directory:\nwandb.save('model.h5')\n\n# Save all files that currently exist containing the substring \"ckpt\":\nwandb.save('../logs/*ckpt*')\n\n# Save any files starting with \"checkpoint\" as they're written to:\nwandb.save(os.path.join(wandb.run.dir, \"checkpoint*\"))\n```", "matching_document_document_id": "e7d893d5-4eae-408f-82f7-2a037a61b392", "query_weight": 0.7432258064516128 }, { "query_id": "a09113c1-8ac2-4adc-9fb1-fc38a03ef476", "query": "Best practices for managing data artifacts in W&B", "matching_document": "## Storage\n\nIf you are approaching or exceeding your storage limit, there are multiple paths forward to manage your data. The path that's best for you will depend on your account type and your current project setup. \n\n## Manage storage consumption \n\nW&B offers different methods of optimizing your storage consumption: \n\n* Use\u00a0reference artifacts\u00a0to track files saved outside the W&B system, instead of uploading them to W&B storage.\n* Use an external cloud storage bucket for storage. *(Enterprise only)* \n\n## Delete data \n\nYou can also choose to delete data to remain under your storage limit. There are several ways to do this: \n\n* Delete data interactively with the app UI.\n* Set a TTL policy on Artifacts so they are automatically deleted.", "matching_document_document_id": "6dea2d7d-58f2-4830-8dc1-27b3bf719859", "query_weight": 0.1 }, { "query_id": "9a124dd2-4ae4-4dd3-8c76-7ca713f34a68", "query": "hyperparameter tuning", "matching_document": "## What Are Hyperparameters?\nWhat\u2019s even more troublesome is that machine learning models are very sensitive to their hyperparameter configurations. The performance of a machine learning model with a certain hyperparameter configuration may not be similar when the hyperparameter configuration is changed. \n\nTo address these problems, we resort to hyperparameter tuning. The general process of this is roughly: \n\n* Select the hyperparameters to be tuned (there can be several hyperparameters in a machine learning model).\n* Specify a grid of acceptable values for the specified hyperparameters or specify distributions that would generate the acceptable values.\n* Train several machine learning models pertaining to each of the different hyperparameter configurations results from the above two steps.\n* Select the model that performs the best from the pool of many models. \n\nAlthough there are many niche techniques that help us in effectively tuning the hyperparameters, the most predominant ones are: \n\n* Grid search\n* Random search", "matching_document_document_id": "4048cd16-70a5-45d9-b6fb-ba7b90e61fca", "query_weight": 0.8825806451612903 }, { "query_id": "22eaec09-07dd-4b5b-a1ef-a039d8dfd42f", "query": "versioning datasets in W&B", "matching_document": "## Dataset Versioning\n### Core Artifacts features\n1. **Upload**: Start tracking and versioning any data (files or directories) with `run.log_artifact()`. You can also track datasets in a remote filesystem (e.g. cloud storage in S3 or GCP) by reference, using a link or URI instead of the raw contents.\n2. **Version**: Define an artifact by giving it a type (`\"raw_data\"`, `\"preprocessed_data\"`, `\"balanced_data\"`) and a name (`\"imagenet_cats_10K\"`). When you log the same name again, W&B automatically creates a new version of the artifact with the latest contents.\n3. **Alias**: Set an alias like `\"best\"` or `\"production\"` to highlight the important versions in a lineage of artifacts.\n4. **Compare**: Select any two versions to browse the contents side-by-side. We're also working on a tool for dataset visualization, learn more here.\n5. **Download**: Obtain a local copy of the artifact or verify the contents by reference. \n\nFor more detail on these features, check out Artifacts how it works.", "matching_document_document_id": "7407e418-a665-438f-b605-d5dc12e4240e", "query_weight": 0.1 }, { "query_id": "0517fc00-1d3c-41fa-b45e-15ff00e10103", "query": "wandb init best practices", "matching_document": "## \u270d\ufe0f W&B Best Practices\n\n1. **Projects**: Log multiple runs to a project to compare them. `wandb.init(project=\"project-name\")`\n2. **Groups**: For multiple processes or cross validation folds, log each process as a runs and group them together. `wandb.init(group=\"experiment-1\")`\n3. **Tags**: Add tags to track your current baseline or production model.\n4. **Notes**: Type notes in the table to track the changes between runs.\n5. **Reports**: Take quick notes on progress to share with colleagues and make dashboards and snapshots of your ML projects.", "matching_document_document_id": "aa2dbfa2-471e-4ec7-b1a2-60a608aa6873", "query_weight": 1.1148387096774193 }, { "query_id": "f3a4cd72-3c85-4c8c-8012-b378f8f41a81", "query": "How to use IAM roles with SageMaker for training job access control?", "matching_document": "## Set up for SageMaker\n### Prerequisites\n1. **Setup SageMaker in your AWS account.** See the SageMaker Developer guide for more information.\n2. **Create an Amazon ECR repository** to store images you want to execute on Amazon SageMaker. See the Amazon ECR documentation for more information.\n3. **Create an Amazon S3 buckets** to store SageMaker inputs and outputs for your SageMaker training jobs. See the Amazon S3 documentation for more information. Make note of the S3 bucket URI and directory.\n4. **Create IAM execution role.** The role used in the SageMaker training job requires the following permissions to work. These permissions allow for logging events, pulling from ECR, and interacting with input and output buckets. (Note: if you already have this role for SageMaker training jobs, you do not need to create it again.)\nIAM role policy\n```\n{\n\"Version\": \"2012-10-17\",\n\"Statement\": [\n{\n\"Effect\": \"Allow\",\n\"Action\": [\n\"cloudwatch:PutMetricData\",\n\"logs:CreateLogStream\",\n\"logs:PutLogEvents\",\n\"logs:CreateLogGroup\",\n\"logs:DescribeLogStreams\",\n\"ecr:GetAuthorizationToken\"\n],\n\"Resource\": \"\\*\"\n},\n{\n\"Effect\": \"Allow\",\n\"Action\": [\n\"s3:ListBucket\"\n],\n\"Resource\": [\n\"arn:aws:s3:::\"\n]\n},\n{\n\"Effect\": \"Allow\",\n\"Action\": [\n\"s3:GetObject\",\n\"s3:PutObject\"\n],\n\"Resource\": [\n\"arn:aws:s3:::/\",\n\"arn:aws:s3:::/\"\n]\n},\n{\n\"Effect\": \"Allow\",\n\"Action\": [\n\"ecr:BatchCheckLayerAvailability\",\n\"ecr:GetDownloadUrlForLayer\",\n\"ecr:BatchGetImage\"\n],\n\"Resource\": \"arn:aws:ecr:::repository/\"\n}\n]\n}\n\n```", "matching_document_document_id": "455be611-540d-4ed2-a34d-bd233f9e98d6", "query_weight": 1.0683870967741935 }, { "query_id": "24d9f9d8-8844-4c5c-b389-251abfcfbd26", "query": "Weights & Biases dashboard features", "matching_document": "## Tutorial\n### Dashboards\nNow we can look at the results. The run we have executed is now shown on the left side, in our project, with the group and experiment names we listed. We have access to a lot of information that W&B has automatically recorded. \n\nWe have several sections like: \n\n* Charts - contains information about losses, accuracy, etc. Also, it contains some examples from our data.\n* System - contains system load information: memory usage, CPU utilization, GPU temp, etc. This is very useful information because you can control the usage of your GPU and choose the optimal batch size.\n* Model - contains information about our model structure (graph).\n* Logs - include Keras default logging.\n* Files - contains all files that were created during the experiment, such as: config, best model, output logs, requirements, etc. The requirements file is very important because, in order to recreate a specific experiment, you need to install specific versions of the libraries.", "matching_document_document_id": "20626155-fa19-4c65-a2d3-9f80b3e7f93a", "query_weight": 0.1 }, { "query_id": "185d5989-2c39-4375-b9cd-7dfa024a502f", "query": "wandb integration with OpenAI", "matching_document": "## OpenAI API\n\nUse the Weights & Biases OpenAI API integration to log requests, responses, token counts and model metadata with 1 line of code for all OpenAI models, including fine-tuned models. \n\n:::info\nThe W&B autolog integration works with `openai <= 0.28.1`. Install the correct version of `openai` with `pip install openai==0.28.1`.\n::: \n\n**Try in a Colab Notebook here \u2192** \n\nWith just 1 line of code you can now automatically log inputs and outputs from the OpenAI Python SDK to Weights & Biases! \n\nOnce you start logging your API inputs and outputs you can quickly evaluate the performance of difference prompts, compare different model settings (such as temperature), and track other usage metrics such as token usage. \n\nTo get started, pip install the `wandb` library, then follow the steps below: \n\n### 1. Import autolog and initialise it \n\nFirst, import `autolog` from `wandb.integration.openai` and initialise it. \n\n### 2. Call the OpenAI API \n\n### 3. View your OpenAI API inputs and responses", "matching_document_document_id": "04695bcc-0c96-4e1e-baf7-113b1b4dca09", "query_weight": 1.1148387096774193 }, { "query_id": "092105c9-c876-45ac-bca0-0b8f0027e350", "query": "Best practices for tracking LLM experiments in Weights & Biases", "matching_document": "### Using Weights & Biases to track experiments\n\nExperimenting with prompts, function calling and response model schema is critical to get good results. As LLM Engineers, we will be methodical and use Weights & Biases to track our experiments. \n\nHere are a few things you should consider logging: \n\n1. Save input and output pairs for later analysis\n2. Save the JSON schema for the response\\_model\n3. Having snapshots of the model and data allow us to compare results over time, and as we make changes to the model we can see how the results change. \n\nThis is particularly useful when we might want to blend a mix of synthetic and real data to evaluate our model. We will use the `wandb` library to track our experiments and save the results to a dashboard.", "matching_document_document_id": "52319db1-144c-48fc-b4ac-eaf148a6815b", "query_weight": 1.207741935483871 }, { "query_id": "5b3973b1-1720-41dc-97a1-edb081dce61c", "query": "How does artifact versioning work in W&B?", "matching_document": "## Dataset Versioning\n### Core Artifacts features\n1. **Upload**: Start tracking and versioning any data (files or directories) with `run.log_artifact()`. You can also track datasets in a remote filesystem (e.g. cloud storage in S3 or GCP) by reference, using a link or URI instead of the raw contents.\n2. **Version**: Define an artifact by giving it a type (`\"raw_data\"`, `\"preprocessed_data\"`, `\"balanced_data\"`) and a name (`\"imagenet_cats_10K\"`). When you log the same name again, W&B automatically creates a new version of the artifact with the latest contents.\n3. **Alias**: Set an alias like `\"best\"` or `\"production\"` to highlight the important versions in a lineage of artifacts.\n4. **Compare**: Select any two versions to browse the contents side-by-side. We're also working on a tool for dataset visualization, learn more here.\n5. **Download**: Obtain a local copy of the artifact or verify the contents by reference. \n\nFor more detail on these features, check out Artifacts how it works.", "matching_document_document_id": "7407e418-a665-438f-b605-d5dc12e4240e", "query_weight": 0.1 }, { "query_id": "f3ecf203-acff-49a8-a9e1-f697ee23fc39", "query": "Can W&B reports be made public?", "matching_document": "--- \n\n## description: Collaborate and share W&B Reports with peers, co-workers, and your team. \n\n# Collaborate on reports \n\nCollaborate and Share W&B Reports \n\nOnce you have saved a report, you can select the **Share** button to collaborate. A draft copy of the report is created when you select the **Edit** button. Draft reports auto-save. Select **Save to report** to publish your changes to the shared report. \n\nA warning notification will appear if an edit conflict occurs. This can occur if you and another collaborator edit the same report at the same time. The warning notification will guide you to resolve potential edit conflicts. \n\n### Comment on reports \n\nClick the comment button on a panel in a report to add a comment directly to that panel. \n\n### Who can edit and share reports? \n\nReports that are created within an individual's private project is only visible to that user. The user can share their project to a team or to the public.", "matching_document_document_id": "7fe941de-f8fe-4acc-b460-004c8b6a298e", "query_weight": 0.6038709677419355 }, { "query_id": "4526c55a-a0c7-49b0-a7c2-d414c0505b43", "query": "W&B artifacts deletion", "matching_document": "## Delete artifacts\n:::note\nArtifacts that are scheduled for deletion with a TTL policy, deleted with the W&B SDK, or deleted with the W&B App UI are first soft-deleted. Artifacts that are soft deleted undergo garbage collection before they are hard-deleted.\n::: \n\n### Delete an artifact version \n\nTo delete an artifact version: \n\n1. Select the name of the artifact. This will expand the artifact view and list all the artifact versions associated with that artifact.\n2. From the list of artifacts, select the artifact version you want to delete.\n3. On the right hand side of the workspace, select the kebab dropdown.\n4. Choose Delete. \n\nAn artifact version can also be deleted programatically via the delete() method. See the examples below. \n\n### Delete multiple artifact versions with aliases \n\nThe following code example demonstrates how to delete artifacts that have aliases associated with them. Provide the entity, project name, and run ID that created the artifacts. \n\n### Delete multiple artifact versions with a specific alias", "matching_document_document_id": "f4d54671-aed9-4d97-96af-8780b8507acd", "query_weight": 1.3935483870967742 }, { "query_id": "2bdd7895-220b-4276-a91c-8991cc0d217d", "query": "Weights & Biases GPU utilization", "matching_document": "## Monitor & Improve GPU Usage for Model Training\n#### 1. Measure your GPU usage consistently over your entire training runs\nYou can\u2019t improve GPU usage without measuring it. It\u2019s not hard to take a snapshot of your usage with useful tools like nvidia-smi, but a simple way to find issues is to track usage over time. Anyone can turn on system monitoring in the background, which will track GPU, CPU, memory usage etc over time by adding two lines to their code: \n\n```\nimport wandb\nwandb.init()\n``` \n\nThe wandb.init() function will create a lightweight child process that will collect system metrics and send them to a wandb server where you can look at them and compare across runs with graphs like these: \n\nThe danger of taking a single measurement is that GPU usage can change over time. This is a common pattern we see where our user Boris is training an RNN; mid-training, his usage plummets from 80 percent to around 25 percent.", "matching_document_document_id": "f57cb60e-1cde-4855-a5e2-1688ac4755e5", "query_weight": 1.207741935483871 }, { "query_id": "10e0156c-c62b-4d8e-9051-49b0f8421ee6", "query": "YOLOv5 Weights & Biases integration guide", "matching_document": "## \ud83d\udce6 YOLOv5 +\ud83e\ude84 W&B\n\nTo install YOLOv5 with W&B integration all we need to is simply [clone this repo](https://github.com/awsaf49/yolov5-wandb). This s repo is synced with official YOLOv5 repo so you'll get the latest features of YOLOv5 along with support of W&B. \n\n> \n\n```\n!git clone https://github.com/awsaf49/yolov5-wandb.git yolov5 # clone\n%cd yolov5\n%pip install -qr requirements.txt # install\n\nimport torch\nimport utils\ndisplay = utils.notebook_init() # checks\n```", "matching_document_document_id": "7d15ab04-df03-4e91-bf66-37d06f5577e3", "query_weight": 0.1 }, { "query_id": "0eab6b7f-f7c5-44e4-86ba-ef702d8a9518", "query": "wandb features", "matching_document": "## Weights & Biases overview\nW&B is a platform that helps data scientists track their models, datasets, system information and more. With a few lines of code, you can start tracking everything about these features. It's free for personal use. Team use is normally a paid utility, but teams for academic purposes are free. You can use W&B with your favourite framework, like TensorFlow, Keras, PyTorch, Sklearn, fastai and many others. \n\nAll tracking information is sent to a dedicated project page on the W&B UI, where you can open high quality visualizations, aggregate information and compare models or parameters. One of the advantages of remotely storing the experiment\u2019s information is that it is easy to collaborate on the same project and share the results with your teammates. \n\nW&B provides 4 useful tools: \n\n* Dashboard: Experiment tracking\n* Artifacts: Dataset versioning, model versioning\n* Sweeps: Hyperparameter optimization\n* Reports: Save and share reproducible findings", "matching_document_document_id": "8c1600ad-9b6e-45cd-86ae-03afd8f80332", "query_weight": 1.1148387096774193 }, { "query_id": "7f410b9b-06d1-4126-a31f-b5f1439b2b13", "query": "Best practices for managing artifacts in W&B", "matching_document": "## Dagster\n### Best practices\n1. Use the IO Manager to read and write Artifacts.\nYou should never need to use `Artifact.download()` or `Run.log_artifact()` directly. Those methods are handled by integration. Simply return the data you wish to store in Artifact and let the integration do the rest. This will provide better lineage for the Artifact in W&B.\n2. Only build an Artifact object yourself for complex use cases.\nPython objects and W&B objects should be returned from your ops/assets. The integration handles bundling the Artifact.\nFor complex use cases, you can build an Artifact directly in a Dagster job. We recommend you pass an Artifact object to the integration for metadata enrichment such as the source integration name and version, the python version used, the pickle protocol version and more.\n3. Add files, directories and external references to your Artifacts through the metadata.\nUse the integration `wandb_artifact_configuration` object to add any file, directory or external references (Amazon S3, GCS, HTTP\u2026). See the advanced example in the Artifact configuration section for more information.\n4. Use an @asset instead of an @op when an Artifact is produced.\nArtifacts are assets. It is recommended to use an asset when Dagster maintains that asset. This will provide better observability in the Dagit Asset Catalog.\n5. Use a SourceAsset to consume an Artifact created outside Dagster.\nThis allows you to take advantage of the integration to read externally created Artifacts. Otherwise, you can only use Artifacts created by the integration.\n6. Use W&B Launch to orchestrate training on dedicated compute for large models.\nYou can train small models inside your Dagster cluster and you can run Dagster in a Kubernetes cluster with GPU nodes. We recommend using W&B Launch for large model training. This will prevent overloading your instance and provide access to more adequate compute.\n7. When experiment tracking within Dagster, set your W&B Run ID to the value of your Dagster Run ID.\nWe recommend that you both: make the Run resumable and set the W&B Run ID to the Dagster Run ID or to a string of your choice. Following this recommendation ensures your W&B metrics and W&B Artifacts are stored in the same W&B Run when you train models inside of Dagster.", "matching_document_document_id": "529f40a6-ab71-4380-8aeb-5ab5335361ec", "query_weight": 1.3935483870967742 }, { "query_id": "6586068a-9f59-474f-995b-dfa5ab7ca73b", "query": "Weights & Biases artifact versioning", "matching_document": "## Create new artifact versions\n### Create new artifact versions from scratch\n* **Single run**: A single run provides all the data for a new version. This is the most common case and is best suited when the run fully recreates the needed data. For example: outputting saved models or model predictions in a table for analysis.\n* **Distributed runs**: A set of runs collectively provides all the data for a new version. This is best suited for distributed jobs which have multiple runs generating data, often in parallel. For example: evaluating a model in a distributed manner, and outputting the predictions. \n\nW&B will create a new artifact and assign it a `v0` alias if you pass a name to the `wandb.Artifact` API that does not exist in your project. W&B checksums the contents when you log again to the same artifact. If the artifact changed, W&B saves a new version `v1`. \n\nW&B will retrieve an existing artifact if you pass a name and artifact type to the `wandb.Artifact` API that matches an existing artifact in your project. The retrieved artifact will have a version greater than 1.", "matching_document_document_id": "2e896de8-7ee0-41ba-983c-2ca7b0b7cb6e", "query_weight": 0.1 }, { "query_id": "33422ea3-ecbb-4f32-9abd-f8e441c6afd7", "query": "What is the configuration file or environment variable to set the artifact cache directory in W&B?", "matching_document": "### Optional Environment Variables\n| Variable name | Usage |\n| --- | --- |\n| **WANDB\\_ANONYMOUS** | Set this to \"allow\", \"never\", or \"must\" to let users create anonymous runs with secret urls. |\n| **WANDB\\_API\\_KEY** | Sets the authentication key associated with your account. You can find your key on your settings page. This must be set if `wandb login` hasn't been run on the remote machine. |\n| **WANDB\\_BASE\\_URL** | If you're using wandb/local you should set this environment variable to `http://YOUR_IP:YOUR_PORT` |\n| **WANDB\\_CACHE\\_DIR** | This defaults to \\~/.cache/wandb, you can override this location with this environment variable |\n| **WANDB\\_CONFIG\\_DIR** | This defaults to \\~/.config/wandb, you can override this location with this environment variable |\n| **WANDB\\_CONFIG\\_PATHS** | Comma separated list of yaml files to load into wandb.config. See config. |\n| **WANDB\\_CONSOLE** | Set this to \"off\" to disable stdout / stderr logging. This defaults to \"on\" in environments that support it. |\n| **WANDB\\_DIR** | Set this to an absolute path to store all generated files here instead of the *wandb* directory relative to your training script. *be sure this directory exists and the user your process runs as can write to it* |\n| **WANDB\\_DISABLE\\_GIT** | Prevent wandb from probing for a git repository and capturing the latest commit / diff. |\n| **WANDB\\_DISABLE\\_CODE** | Set this to true to prevent wandb from saving notebooks or git diffs. We'll still save the current commit if we're in a git repo. |\n| **WANDB\\_DOCKER** | Set this to a docker image digest to enable restoring of runs. This is set automatically with the wandb docker command. You can obtain an image digest by running `wandb docker my/image/name:tag --digest` |", "matching_document_document_id": "ab26d105-0c7e-46c4-949a-f8e9ca33bf3d", "query_weight": 1.3935483870967742 }, { "query_id": "efde5c0a-582a-46b9-a3ff-333833e43044", "query": "W&B logging features", "matching_document": "## Managing and Tracking ML Experiments With W&B\n### Logging Advanced\u00a0Things\u200b\n\nOne of the coolest things about W&B is that you can literally log anything. You can log custom metrics, matplotlib plots, datasets, embeddings from your models, prediction distribution, etc. \n\nRecently, Weights & Biases announced the Tables feature, which allows you to log, query and analyze tabular data. You can even visualize model predictions and compare them across models. For example: see the image below (taken from[ W&B Docs](http://docs.wandb.ai)), which compares two segmentation models. \n\nYou can log audio data, images, histograms, text, video, and tabular data and visualize/inspect them interactively. To learn more about W&B Tables, go through their [documentation](https://docs.wandb.ai/guides/data-vis). \n\nYou can even export the dashboard in CSV files to analyze them further. W&B also supports exports in PNG, SVG, PDF, and CSV, depending on the type of data you are trying to export.", "matching_document_document_id": "5b04907e-a04e-40a7-939f-b745aa031559", "query_weight": 1.2541935483870967 }, { "query_id": "96fdfcf0-dec7-4df0-97ad-68b7969dcaf3", "query": "Weights & Biases integration with LangChain", "matching_document": "## Prompts Quickstart\n### Use W&B Trace with LangChain\n#### 1. Set the LANGCHAIN\\_WANDB\\_TRACING environment variable\nFirst, set the LANGCHAIN\\_WANDB\\_TRACING environment variable to true. This will turn on automated Weights & Biases logging with LangChain: \n\n```\nimport os\n\n# turn on wandb logging for langchain\nos.environ[\"LANGCHAIN\\_WANDB\\_TRACING\"] = \"true\"\n\n``` \n\nThats it! Now any call to a LangChain LLM, Chain, Tool or Agent will be logged to Weights & Biases.", "matching_document_document_id": "6adad621-230b-42f0-badc-1738fef555a6", "query_weight": 1.2541935483870967 }, { "query_id": "c50a98ea-bbd5-4007-97af-d0db7a25b2b7", "query": "What are the best practices for using Weights & Biases in deep learning projects?", "matching_document": "## \u270d\ufe0f W&B Best Practices\n\n1. **Projects**: Log multiple runs to a project to compare them. `wandb.init(project=\"project-name\")`\n2. **Groups**: For multiple processes or cross validation folds, log each process as a runs and group them together. `wandb.init(group=\"experiment-1\")`\n3. **Tags**: Add tags to track your current baseline or production model.\n4. **Notes**: Type notes in the table to track the changes between runs.\n5. **Reports**: Take quick notes on progress to share with colleagues and make dashboards and snapshots of your ML projects.", "matching_document_document_id": "aa2dbfa2-471e-4ec7-b1a2-60a608aa6873", "query_weight": 1.2541935483870967 }, { "query_id": "399c542d-f4b5-45b8-84fb-4bc8f26d1f78", "query": "Overview of Weights & Biases features", "matching_document": "## Weights & Biases overview\nW&B is a platform that helps data scientists track their models, datasets, system information and more. With a few lines of code, you can start tracking everything about these features. It's free for personal use. Team use is normally a paid utility, but teams for academic purposes are free. You can use W&B with your favourite framework, like TensorFlow, Keras, PyTorch, Sklearn, fastai and many others. \n\nAll tracking information is sent to a dedicated project page on the W&B UI, where you can open high quality visualizations, aggregate information and compare models or parameters. One of the advantages of remotely storing the experiment\u2019s information is that it is easy to collaborate on the same project and share the results with your teammates. \n\nW&B provides 4 useful tools: \n\n* Dashboard: Experiment tracking\n* Artifacts: Dataset versioning, model versioning\n* Sweeps: Hyperparameter optimization\n* Reports: Save and share reproducible findings", "matching_document_document_id": "8c1600ad-9b6e-45cd-86ae-03afd8f80332", "query_weight": 0.9754838709677419 }, { "query_id": "597c4228-16a2-432e-9e8e-d9e9a672299b", "query": "Examples of logging images in Wandb", "matching_document": "## Log Media & Objects\n### Images\n```\nimages = wandb.Image(image\\_array, caption=\"Top: Output, Bottom: Input\")\n\nwandb.log({\"examples\": images})\n\n``` \n\nWe assume the image is gray scale if the last dimension is 1, RGB if it's 3, and RGBA if it's 4. If the array contains floats, we convert them to integers between `0` and `255`. If you want to normalize your images differently, you can specify the `mode` manually or just supply a `PIL.Image`, as described in the \"Logging PIL Images\" tab of this panel. \n\nFor full control over the conversion of arrays to images, construct the `PIL.Image` yourself and provide it directly. \n\n```\nimages = [PIL.Image.fromarray(image) for image in image\\_array]\n\nwandb.log({\"examples\": [wandb.Image(image) for image in images]})\n\n``` \n\nFor even more control, create images however you like, save them to disk, and provide a filepath. \n\n```\nim = PIL.fromarray(...)\nrgb\\_im = im.convert(\"RGB\")\nrgb\\_im.save(\"myimage.jpg\")\n\nwandb.log({\"example\": wandb.Image(\"myimage.jpg\")})\n\n```", "matching_document_document_id": "cefd3fca-d245-469d-95bf-eb8b4b7f28f9", "query_weight": 0.9754838709677419 }, { "query_id": "c5f70f42-cd07-44a5-904e-96256dbf21ce", "query": "Artifact versioning in Weights & Biases with S3 datasets", "matching_document": "## Dataset Versioning\n### Core Artifacts features\n1. **Upload**: Start tracking and versioning any data (files or directories) with `run.log_artifact()`. You can also track datasets in a remote filesystem (e.g. cloud storage in S3 or GCP) by reference, using a link or URI instead of the raw contents.\n2. **Version**: Define an artifact by giving it a type (`\"raw_data\"`, `\"preprocessed_data\"`, `\"balanced_data\"`) and a name (`\"imagenet_cats_10K\"`). When you log the same name again, W&B automatically creates a new version of the artifact with the latest contents.\n3. **Alias**: Set an alias like `\"best\"` or `\"production\"` to highlight the important versions in a lineage of artifacts.\n4. **Compare**: Select any two versions to browse the contents side-by-side. We're also working on a tool for dataset visualization, learn more here.\n5. **Download**: Obtain a local copy of the artifact or verify the contents by reference. \n\nFor more detail on these features, check out Artifacts how it works.", "matching_document_document_id": "7407e418-a665-438f-b605-d5dc12e4240e", "query_weight": 1.3006451612903225 }, { "query_id": "ad1e2a8b-2401-4f9f-b586-7ebd2e2e9a12", "query": "Weights & Biases artifacts documentation", "matching_document": "Arguments:\n name: A human-readable name for the artifact. Use the name to identify\n a specific artifact in the W&B App UI or programmatically. You can\n interactively reference an artifact with the `use_artifact` Public API.\n A name can contain letters, numbers, underscores, hyphens, and dots.\n The name must be unique across a project.\n type: The artifact's type. Use the type of an artifact to both organize\n and differentiate artifacts. You can use any string that contains letters,\n numbers, underscores, hyphens, and dots. Common types include `dataset` or `model`.\n Include `model` within your type string if you want to link the artifact\n to the W&B Model Registry.\n description: A description of the artifact. For Model or Dataset Artifacts,\n add documentation for your standardized team model or dataset card. View\n an artifact's description programmatically with the `Artifact.description`\n attribute or programmatically with the W&B App UI. W&B renders the\n description as markdown in the W&B App.\n metadata: Additional information about an artifact. Specify metadata as a\n dictionary of key-value pairs. You can specify no more than 100 total keys.\n\n Returns:\n An `Artifact` object.\n \"\"\"", "matching_document_document_id": "be3722cf-110a-46ac-a0e1-58aba5de5745", "query_weight": 0.5109677419354839 }, { "query_id": "a9077f1a-f73d-4f65-9685-218ca73fc760", "query": "W&B team collaboration", "matching_document": "## Teams\n\nUse W&B Teams as a central workspace for your ML team to build better models faster. \n\n* **Track all the experiments** your team has tried so you never duplicate work.\n* **Save and reproduce** previously trained models.\n* **Share progress** and results with your boss and collaborators.\n* **Catch regressions** and immediately get alerted when performance drops.\n* **Benchmark model performance** and compare model versions. \n\n## Create a collaborative team \n\n1. **Sign up or log in** to your free W&B account.\n2. Click **Invite Team** in the navigation bar.\n3. Create your team and invite collaborators. \n\n:::info\n**Note**: Only the admin of an organization can create a new team.\n::: \n\n## Invite team members \n\n## Create a Team Profile \n\nYou can customize your team's profile page to show an introduction and showcase reports and projects that are visible to the public or team members. Present reports, projects, and external links. \n\n## Remove team members \n\n## Team Roles and Permissions", "matching_document_document_id": "d2f20931-cbc4-48cf-9c3e-4c52be6349b2", "query_weight": 1.3470967741935482 }, { "query_id": "767786b8-a725-486d-ae0a-a7766f09bb38", "query": "W&B model versioning", "matching_document": "## Terms and concepts\n\nModel Registry terms and concepts \n\nThe following terms describe key components of the W&B Model Registry: *model version*, *model artifact*, and *registered model*. \n\n## Model version \n\nA model version represents a single model checkpoint. Model versions are a snapshot at a point in time of a model and its files within an experiment. \n\nA model version is an immutable directory of data and metadata that describes a trained model. W&B suggests that you add files to your model version that let you store (and restore) your model architecture and learned parameters at a later date. \n\nA model version belongs to one, and only one, model artifact. A model version can belong to zero or more, registered models. Model versions are stored in a model artifact in the order they are logged to the model artifact. W&B automatically creates a new model version if it detects that a model you log (to the same model artifact) has different contents than a previous model version. \n\n## Model alias", "matching_document_document_id": "93d77de6-8c4d-4720-8f2f-a781d22a75be", "query_weight": 0.1 }, { "query_id": "6e80e67d-296e-4096-96f9-2ad408caccac", "query": "Weights & Biases", "matching_document": "## What is W&B?\n\nWeights & Biases (W&B) is the AI developer platform, with tools for training models, fine-tuning models, and leveraging foundation models. \n\nSet up W&B in 5 minutes, then quickly iterate on your machine learning pipeline with the confidence that your models and data are tracked and versioned in a reliable system of record. \n\nThis diagram outlines the relationship between W&B products. \n\n**W&B Models** is a set of lightweight, interoperable tools for machine learning practitioners training and fine-tuning models.\n- Experiments: Machine learning experiment tracking\n- Model Registry: Manage production models centrally\n- Launch: Scale and automate workloads\n- Sweeps: Hyperparameter tuning and model optimization \n\n**W&B Prompts** is for LLM debugging and monitoring, including usage of OpenAI's GPT API.", "matching_document_document_id": "7178c033-7049-4ab8-b4bb-2d7b71a495a8", "query_weight": 0.9754838709677419 }, { "query_id": "3911dd96-0114-4179-adc0-ce9126a9568e", "query": "How to structure Weights & Biases runs for hyperparameter tuning?", "matching_document": "## Whats Next? Hyperparameters with Sweeps\n\nWe tried out two different hyperparameter settings by hand. You can use Weights & Biases Sweeps to automate hyperparameter testing and explore the space of possible models and optimization strategies. \n\n## Check out Hyperparameter Optimization in TensorFlow uisng W&B Sweep $\\rightarrow$ \n\nRunning a hyperparameter sweep with Weights & Biases is very easy. There are just 3 simple steps: \n\n1. **Define the sweep:** We do this by creating a dictionary or a YAML file that specifies the parameters to search through, the search strategy, the optimization metric et all.\n2. **Initialize the sweep:**\n`sweep_id = wandb.sweep(sweep_config)`\n3. **Run the sweep agent:**\n`wandb.agent(sweep_id, function=train)` \n\nAnd voila! That's all there is to running a hyperparameter sweep! In the notebook below, we'll walk through these 3 steps in more detail.", "matching_document_document_id": "c2c3cc54-f20f-40b0-9fdc-19115bb311ba", "query_weight": 1.3470967741935482 }, { "query_id": "01f7c67c-9125-45a6-a149-4ad7eb671f88", "query": "machine learning model tracking", "matching_document": "## Track a model\n\nTrack a model, the model's dependencies, and other information relevant to that model with the W&B Python SDK. \n\nUnder the hood, W&B creates a lineage of model artifact that you can view with the W&B App UI or programmatically with the W&B Python SDK. See the Create model lineage map for more information. \n\n## How to log a model \n\nUse the `run.log_model` API to log a model. Provide the path where your model files are saved to the `path` parameter. The path can be a local file, directory, or reference URI to an external bucket such as `s3://bucket/path`. \n\nOptionally provide a name for the model artifact for the `name` parameter. If `name` is not specified, W&B uses the basename of the input path prepended with the run ID. \n\nCopy and paste the proceeding code snippet. Ensure to replace values enclosed in `<>` with your own.", "matching_document_document_id": "fdcd64f1-b177-42e2-88b2-6df060ec5b67", "query_weight": 0.1 }, { "query_id": "8c55b62f-ce98-4d00-acd3-76057644bdd7", "query": "saving files with wandb", "matching_document": "## Save & Restore Files\n### Saving Files\n#### Examples of `wandb.save`\n```\n# Save a model file from the current directory\nwandb.save(\"model.h5\")\n\n# Save all files that exist containing the substring \"ckpt\"\nwandb.save(\"../logs/\\*ckpt\\*\")\n\n# Save files starting with \"checkpoint\" as they're written to\nwandb.save(os.path.join(wandb.run.dir, \"checkpoint\\*\"))\n\n``` \n\n:::info\nW&B's local run directories are by default inside the `./wandb` directory relative to your script, and the path looks like `run-20171023_105053-3o4933r0` where `20171023_105053` is the timestamp and `3o4933r0` is the ID of the run. You can set the `WANDB_DIR` environment variable, or the `dir` keyword argument of `wandb.init`, to an absolute path and files will be written within that directory instead.\n:::", "matching_document_document_id": "0d3b29a7-20a4-489e-a16e-70827196337e", "query_weight": 0.7432258064516128 }, { "query_id": "20eceb0d-f141-4315-9f96-3ead46c4b8fd", "query": "YOLOv5 integration", "matching_document": "## \ud83d\udce6 YOLOv5 +\ud83e\ude84 W&B\n\nTo install YOLOv5 with W&B integration all we need to is simply [clone this repo](https://github.com/awsaf49/yolov5-wandb). This s repo is synced with official YOLOv5 repo so you'll get the latest features of YOLOv5 along with support of W&B. \n\n> \n\n```\n!git clone https://github.com/awsaf49/yolov5-wandb.git yolov5 # clone\n%cd yolov5\n%pip install -qr requirements.txt # install\n\nimport torch\nimport utils\ndisplay = utils.notebook_init() # checks\n```", "matching_document_document_id": "7d15ab04-df03-4e91-bf66-37d06f5577e3", "query_weight": 0.1 }, { "query_id": "7734ba38-f91b-4899-8097-a50f280f2b58", "query": "Integrating Weights & Biases with Hugging Face's Accelerate", "matching_document": "## Hugging Face Accelerate\n\nAccelerate is a library that enables the same PyTorch code to be run across any distributed configuration by adding just four lines of code, making training and inference at scale made simple, efficient and adaptable. \n\nAccelerate includes a Weights & Biases Tracker which we show how to use below. You can also read more about Accelerate Trackers in **their docs here** \n\n## Start logging with Accelerate \n\nTo get started with Accelerate and Weights & Biases you can follow the pseudocode below: \n\n## Accessing Accelerates' Internal W&B Tracker \n\nYou can quickly access the wandb tracker using the Accelerator.get\\_tracker() method. Just pass in the string corresponding to a tracker\u2019s .name attribute and it will return that tracker on the main process. \n\n```\nwandb\\_tracker = accelerator.get\\_tracker(\"wandb\")\n\n``` \n\nFrom there you can interact with wandb\u2019s run object like normal: \n\n```\nwandb\\_tracker.log\\_artifact(some\\_artifact\\_to\\_log)\n\n``` \n\n## Accelerate Articles", "matching_document_document_id": "4f08b4a7-bc90-4cf4-b3e4-d6224fde90fd", "query_weight": 0.1 }, { "query_id": "63fde3c5-38e2-497d-914f-b338b2f6d338", "query": "Steps to link a run to the model registry in Weights & Biases.", "matching_document": "## W&B Tutorial with Pytorch Lightning\n### Model Registry\n\nAfter logging a bunch of checkpoints across multiple runs during experimentation, now comes time to hand-off the best checkpoint to the next stage of the workflow (e.g. testing, deployment). \n\nThe model registry offers a centralized place to house the best checkpoints for all your model tasks. Any `model` artifact you log can be \"linked\" to a Registered Model. Here are the steps to start using the model registry for more organized model management:\n1. Access your team's model registry by going the team page and selecting `Model Registry` \n\n1. Create a new Registered Model.\n2. Go to the artifacts tab of the project that holds all your model checkpoints\n3. Click \"Link to Registry\" for the model artifact version you want. (Alternatively you can link a model via api with `wandb.run.link_artifact`)", "matching_document_document_id": "cf34b579-6181-4435-87ca-06d3cee2dc38", "query_weight": 0.1 }, { "query_id": "fcafab53-ed41-4fec-9639-d7dc0cd2878d", "query": "How to download artifacts from Weights & Biases?", "matching_document": "## Download and use artifacts\n#### Download and use an artifact stored on W&B\nUse the object returned to download all the contents of the artifact: \n\n```\ndatadir = artifact.download()\n\n``` \n\nYou can optionally pass a path to the root parameter to download the contents of the artifact to a specific directory. For more information, see the Python SDK Reference Guide. \n\nUse the `get_path` method to download only subset of files: \n\n```\npath = artifact.get\\_path(name)\n\n``` \n\nThis fetches only the file at the path `name`. It returns an `Entry` object with the following methods: \n\n* `Entry.download`: Downloads file from the artifact at path `name`\n* `Entry.ref`: If the entry was stored as a reference using `add_reference`, returns the URI \n\nReferences that have schemes that W&B knows how to handle can be downloaded just like artifact files. For more information, see Track external files. \n\nFirst, import the W&B SDK. Next, create an artifact from the Public API Class. Provide the entity, project, artifact, and alias associated with that artifact:", "matching_document_document_id": "a42e4236-eca7-4763-a693-b44e8187ca6f", "query_weight": 1.1148387096774193 }, { "query_id": "3f8b3679-df1b-4e28-8990-94066bd84772", "query": "What are the best practices for using artifacts to manage large datasets in Weights & Biases?", "matching_document": "## Track external files\n### Track artifacts outside of Weights & Biases\nUse Weights & Biases Artifacts for dataset versioning and model lineage, and use **reference artifacts** to track files saved outside the W&B server. In this mode an artifact only stores metadata about the files, such as URLs, size, and checksums. The underlying data never leaves your system. See the Quick start for information on how to save files and directories to W&B servers instead. \n\nFor an example of tracking reference files in GCP, see the Guide to Tracking Artifacts by Reference. \n\nThe following describes how to construct reference artifacts and how to best incorporate them into your workflows. \n\n### Amazon S3 / GCS References \n\nUse Weights & Biases Artifacts for dataset and model versioning to track references in cloud storage buckets. With artifact references, seamlessly layer tracking on top of your buckets with no modifications to your existing storage layout. \n\n### Download a reference artifact \n\n### Tying it together \n\n### Filesystem References", "matching_document_document_id": "677816ff-ebf9-4c97-8d16-0ee17df204ad", "query_weight": 0.1 }, { "query_id": "27a47969-0106-488b-8643-b509fb1e36ca", "query": "Tracking and comparing LLM experiments in Weights & Biases", "matching_document": "### Using Weights & Biases to track experiments\n\nExperimenting with prompts, function calling and response model schema is critical to get good results. As LLM Engineers, we will be methodical and use Weights & Biases to track our experiments. \n\nHere are a few things you should consider logging: \n\n1. Save input and output pairs for later analysis\n2. Save the JSON schema for the response\\_model\n3. Having snapshots of the model and data allow us to compare results over time, and as we make changes to the model we can see how the results change. \n\nThis is particularly useful when we might want to blend a mix of synthetic and real data to evaluate our model. We will use the `wandb` library to track our experiments and save the results to a dashboard.", "matching_document_document_id": "52319db1-144c-48fc-b4ac-eaf148a6815b", "query_weight": 1.207741935483871 }, { "query_id": "fdbc1ed3-11d1-4187-b6a5-d4f595a0c7f6", "query": "log prompts wandb", "matching_document": "def log_index(vector_store_dir: str, run: \"wandb.run\"):\n \"\"\"Log a vector store to wandb\n\n Args:\n vector_store_dir (str): The directory containing the vector store to log\n run (wandb.run): The wandb run to log the artifact to.\n \"\"\"\n index_artifact = wandb.Artifact(name=\"vector_store\", type=\"search_index\")\n index_artifact.add_dir(vector_store_dir)\n run.log_artifact(index_artifact)\n\ndef log_prompt(prompt: dict, run: \"wandb.run\"):\n \"\"\"Log a prompt to wandb\n\n Args:\n prompt (str): The prompt to log\n run (wandb.run): The wandb run to log the artifact to.\n \"\"\"\n prompt_artifact = wandb.Artifact(name=\"chat_prompt\", type=\"prompt\")\n with prompt_artifact.new_file(\"prompt.json\") as f:\n f.write(json.dumps(prompt))\n run.log_artifact(prompt_artifact)", "matching_document_document_id": "6edbec3a-49b0-47a6-91e6-e5484f32aeab", "query_weight": 1.7651612903225806 }, { "query_id": "b47401c0-07c4-4e79-b122-785a09dac059", "query": "How to integrate LangChain with Weights & Biases", "matching_document": "## Prompts Quickstart\n### Use W&B Trace with LangChain\n#### 1. Set the LANGCHAIN\\_WANDB\\_TRACING environment variable\nFirst, set the LANGCHAIN\\_WANDB\\_TRACING environment variable to true. This will turn on automated Weights & Biases logging with LangChain: \n\n```\nimport os\n\n# turn on wandb logging for langchain\nos.environ[\"LANGCHAIN\\_WANDB\\_TRACING\"] = \"true\"\n\n``` \n\nThats it! Now any call to a LangChain LLM, Chain, Tool or Agent will be logged to Weights & Biases.", "matching_document_document_id": "6adad621-230b-42f0-badc-1738fef555a6", "query_weight": 1.2541935483870967 }, { "query_id": "e7b2b6ce-10c6-4488-8ea7-c05aed28e1fd", "query": "exporting data from Weights & Biases charts", "matching_document": "\" \n\n# Export Your Data from W&B \n\nDescription: Learn how to export CSV, PDF, and LaTeX from W&B \n\nBody: \n\nWe're excited to announce our latest highly requested feature! You can now export tables, graphs and reports to CSV, LaTeX and PDF. Here are some examples of how you can export data easily from the Weights & Biases UI. \n\nIn any graph, click on the arrow in the upper right and click \u201cPanel Export\u201d. \n\nYou\u2019ll see a popup table with the underlying data - click Save as CSV to start the CSV download. \n\nIn a table click the \u201cExport Table\u201d button in the upper right corner. \n\nIn a report, click the \u201cDownload as LaTeX\u201d link. \n\nThis will download a zip file with a report.tex and images of your panels attached. The .tex file is a latex file that you can modify to style your document. You can load the zip file into Overleaf.com and use their visual editor, or you can download pdflatex and run \u2018pdflatext report.tex\u2019 to generate a report.pdf file. \n\nHere\u2019s an example of an exported LaTeX report: \n\n\"", "matching_document_document_id": "78c84355-d5cd-4b5f-920d-509ab7bf3ad9", "query_weight": 1.8116129032258064 }, { "query_id": "9862d742-5fb3-4cab-b24d-64df6ef18869", "query": "How can LangChain be used in conjunction with Weights & Biases?", "matching_document": "## Prompts Quickstart\n### Use W&B Trace with LangChain\n#### 1. Set the LANGCHAIN\\_WANDB\\_TRACING environment variable\nFirst, set the LANGCHAIN\\_WANDB\\_TRACING environment variable to true. This will turn on automated Weights & Biases logging with LangChain: \n\n```\nimport os\n\n# turn on wandb logging for langchain\nos.environ[\"LANGCHAIN\\_WANDB\\_TRACING\"] = \"true\"\n\n``` \n\nThats it! Now any call to a LangChain LLM, Chain, Tool or Agent will be logged to Weights & Biases.", "matching_document_document_id": "6adad621-230b-42f0-badc-1738fef555a6", "query_weight": 1.2541935483870967 }, { "query_id": "9cfba31d-ee16-485e-aabd-4357e3506db5", "query": "logging distributed training wandb", "matching_document": "## Experiments FAQ\n#### How can I use wandb with multiprocessing, e.g. distributed training?\n\nIf your training program uses multiple processes you will need to structure your program to avoid making wandb method calls from processes where you did not run `wandb.init()`.\\\n\\\nThere are several approaches to managing multiprocess training: \n\n1. Call `wandb.init` in all your processes, using the group keyword argument to define a shared group. Each process will have its own wandb run and the UI will group the training processes together.\n2. Call `wandb.init` from just one process and pass data to be logged over multiprocessing queues. \n\n:::info\nCheck out the Distributed Training Guide for more detail on these two approaches, including code examples with Torch DDP.\n:::", "matching_document_document_id": "cc461aea-36f0-4a9b-be45-e7110b8cfd88", "query_weight": 0.7432258064516128 }, { "query_id": "5789c72d-9071-461b-9bb0-70ffbe32a6bd", "query": "How to implement AWS IAM authentication in SageMaker training jobs?", "matching_document": "## Set up for SageMaker\n### Prerequisites\n1. **Setup SageMaker in your AWS account.** See the SageMaker Developer guide for more information.\n2. **Create an Amazon ECR repository** to store images you want to execute on Amazon SageMaker. See the Amazon ECR documentation for more information.\n3. **Create an Amazon S3 buckets** to store SageMaker inputs and outputs for your SageMaker training jobs. See the Amazon S3 documentation for more information. Make note of the S3 bucket URI and directory.\n4. **Create IAM execution role.** The role used in the SageMaker training job requires the following permissions to work. These permissions allow for logging events, pulling from ECR, and interacting with input and output buckets. (Note: if you already have this role for SageMaker training jobs, you do not need to create it again.)\nIAM role policy\n```\n{\n\"Version\": \"2012-10-17\",\n\"Statement\": [\n{\n\"Effect\": \"Allow\",\n\"Action\": [\n\"cloudwatch:PutMetricData\",\n\"logs:CreateLogStream\",\n\"logs:PutLogEvents\",\n\"logs:CreateLogGroup\",\n\"logs:DescribeLogStreams\",\n\"ecr:GetAuthorizationToken\"\n],\n\"Resource\": \"\\*\"\n},\n{\n\"Effect\": \"Allow\",\n\"Action\": [\n\"s3:ListBucket\"\n],\n\"Resource\": [\n\"arn:aws:s3:::\"\n]\n},\n{\n\"Effect\": \"Allow\",\n\"Action\": [\n\"s3:GetObject\",\n\"s3:PutObject\"\n],\n\"Resource\": [\n\"arn:aws:s3:::/\",\n\"arn:aws:s3:::/\"\n]\n},\n{\n\"Effect\": \"Allow\",\n\"Action\": [\n\"ecr:BatchCheckLayerAvailability\",\n\"ecr:GetDownloadUrlForLayer\",\n\"ecr:BatchGetImage\"\n],\n\"Resource\": \"arn:aws:ecr:::repository/\"\n}\n]\n}\n\n```", "matching_document_document_id": "455be611-540d-4ed2-a34d-bd233f9e98d6", "query_weight": 1.0683870967741935 }, { "query_id": "09f90ac8-b788-4e79-9e78-1b778e328cff", "query": "logging confusion matrix as chart in wandb", "matching_document": "## Log Plots\n#### Model Evaluation Charts\nYou can log this whenever your code has access to: \n\n* a model's predicted scores (`predictions`) on a set of examples\n* the corresponding ground truth labels (`ground_truth`) for those examples\n* (optionally) a list of the labels/ class names (`labels=[\"cat\", \"dog\", \"bird\"...]` if label index 0 means cat, 1 = dog, 2 = bird, etc.)\n* (optionally) a subset (still in list format) of these labels to visualize on the plot \n\nSee in the app \u2192 \n\nRun the code \u2192 \n\n`wandb.plot.confusion_matrix()` \n\nCreate a multi-class confusion matrix in one line: \n\n```\ncm = wandb.plot.confusion\\_matrix(\ny\\_true=ground\\_truth, preds=predictions, class\\_names=class\\_names\n)\n\nwandb.log({\"conf\\_mat\": cm})\n\n``` \n\nYou can log this wherever your code has access to:", "matching_document_document_id": "ea7c9109-febd-4056-8d23-26cfbc88bec6", "query_weight": 0.8361290322580645 }, { "query_id": "b0c1be8a-e9c4-4726-bf32-2c6ef8ebe75c", "query": "Python code integration with W&B", "matching_document": "## Add W&B to your code\n#### Training script with W&B Python SDK\n1. Line 1: Import the Weights & Biases Python SDK.\n2. Line 6: Create a dictionary object where the key-value pairs define the sweep configuration. In the proceeding example, the batch size (`batch_size`), epochs (`epochs`), and the learning rate (`lr`) hyperparameters are varied during each sweep. For more information on how to create a sweep configuration, see Define sweep configuration.\n3. Line 19: Pass the sweep configuration dictionary to `wandb.sweep`. This initializes the sweep. This returns a sweep ID (`sweep_id`). For more information on how to initialize sweeps, see Initialize sweeps.\n4. Line 33: Use the `wandb.init()` API to generate a background process to sync and log data as a W&B Run.\n5. Line 37-39: (Optional) define values from `wandb.config` instead of defining hard coded values.\n6. Line 45: Log the metric we want to optimize with `wandb.log`. You must log the metric defined in your configuration. Within the configuration dictionary (`sweep_configuration` in this example) we defined the sweep to maximize the `val_acc` value).\n7. Line 54: Start the sweep with the `wandb.agent` API call. Provide the sweep ID (line 19), the name of the function the sweep will execute (`function=main`), and set the maximum number of runs to try to four (`count=4`). For more information on how to start W&B Sweep, see Start sweep agents.", "matching_document_document_id": "ad768026-ec4b-410d-a4c7-ed5565f74532", "query_weight": 1.2541935483870967 }, { "query_id": "98d5212a-0136-4945-b96d-fff31946b9e6", "query": "Securing AWS SageMaker training jobs with IAM roles", "matching_document": "## Set up for SageMaker\n### Prerequisites\n1. **Setup SageMaker in your AWS account.** See the SageMaker Developer guide for more information.\n2. **Create an Amazon ECR repository** to store images you want to execute on Amazon SageMaker. See the Amazon ECR documentation for more information.\n3. **Create an Amazon S3 buckets** to store SageMaker inputs and outputs for your SageMaker training jobs. See the Amazon S3 documentation for more information. Make note of the S3 bucket URI and directory.\n4. **Create IAM execution role.** The role used in the SageMaker training job requires the following permissions to work. These permissions allow for logging events, pulling from ECR, and interacting with input and output buckets. (Note: if you already have this role for SageMaker training jobs, you do not need to create it again.)\nIAM role policy\n```\n{\n\"Version\": \"2012-10-17\",\n\"Statement\": [\n{\n\"Effect\": \"Allow\",\n\"Action\": [\n\"cloudwatch:PutMetricData\",\n\"logs:CreateLogStream\",\n\"logs:PutLogEvents\",\n\"logs:CreateLogGroup\",\n\"logs:DescribeLogStreams\",\n\"ecr:GetAuthorizationToken\"\n],\n\"Resource\": \"\\*\"\n},\n{\n\"Effect\": \"Allow\",\n\"Action\": [\n\"s3:ListBucket\"\n],\n\"Resource\": [\n\"arn:aws:s3:::\"\n]\n},\n{\n\"Effect\": \"Allow\",\n\"Action\": [\n\"s3:GetObject\",\n\"s3:PutObject\"\n],\n\"Resource\": [\n\"arn:aws:s3:::/\",\n\"arn:aws:s3:::/\"\n]\n},\n{\n\"Effect\": \"Allow\",\n\"Action\": [\n\"ecr:BatchCheckLayerAvailability\",\n\"ecr:GetDownloadUrlForLayer\",\n\"ecr:BatchGetImage\"\n],\n\"Resource\": \"arn:aws:ecr:::repository/\"\n}\n]\n}\n\n```", "matching_document_document_id": "455be611-540d-4ed2-a34d-bd233f9e98d6", "query_weight": 1.0683870967741935 }, { "query_id": "5822c07f-25fc-4365-b4cd-fa4c9bbb4a06", "query": "using tables in W&B", "matching_document": "--- \n\n## description: Explore how to use W&B Tables with this 5 minute Quickstart. \n\n# Tables Quickstart \n\nThe following Quickstart demonstrates how to log data tables, visualize data, and query data. \n\nSelect the button below to try a PyTorch Quickstart example project on MNIST data. \n\n## 1. Log a table \n\nFollow the procedure outlined below to log a Table with W&B:\n1. Initialize a W&B Run with `wandb.init()`.\n2. Create a `wandb.Table()` object instance. Pass the name of the columns in your table along with the data for the `columns` and `data` parameters, respectively. \n\n3. Log the table with `run.log()` as a key-value pair. Provide a name for your table for the key, and pass the object instance of `wandb.Table` as the value. \n\n```\nrun = wandb.init(project=\"table-test\")\nmy\\_table = wandb.Table(columns=[\"a\", \"b\"], data=[[\"a1\", \"b1\"], [\"a2\", \"b2\"]])\nrun.log({\"Table Name\": my\\_table})\n\n``` \n\n## 2. Visualize tables in the workspace \n\n## 3. Compare across model versions", "matching_document_document_id": "1c93e1a4-c017-4497-904a-7c76a31c7b7a", "query_weight": 1.2541935483870967 }, { "query_id": "4759b32f-bcc9-4062-8c3c-3ee18098a060", "query": "How to configure a sweep in Weights & Biases without a YAML file?", "matching_document": "## Sweep configuration structure\n### Basic structure\nBoth sweep configuration format options (YAML and Python dictionary) utilize key-value pairs and nested structures. \n\nUse top-level keys within your sweep configuration to define qualities of your sweep search such as the name of the sweep (`name` key), the parameters to search through (`parameters` key), the methodology to search the parameter space (`method` key), and more. \n\nFor example, the proceeding code snippets show the same sweep configuration defined within a YAML file and within a Python dictionary. Within the sweep configuration there are five top level keys specified: `program`, `name`, `method`, `metric` and `parameters`. \n\n{label: 'CLI', value: 'cli'},\n{label: 'Python script or Jupyter notebook', value: 'script'},\n]}> \n\nDefine a sweep in a Python dictionary data structure if you define training algorithm in a Python script or Jupyter notebook. \n\nThe proceeding code snippet stores a sweep configuration in a variable named `sweep_configuration`:", "matching_document_document_id": "69de0aa1-1dad-465e-8e28-55db1bb48e80", "query_weight": 0.1 }, { "query_id": "ac1a6df3-71fc-43ea-855b-f6dca398a39b", "query": "How can LangChain be integrated with Weights & Biases?", "matching_document": "## Prompts Quickstart\n### Use W&B Trace with LangChain\n#### 1. Set the LANGCHAIN\\_WANDB\\_TRACING environment variable\nFirst, set the LANGCHAIN\\_WANDB\\_TRACING environment variable to true. This will turn on automated Weights & Biases logging with LangChain: \n\n```\nimport os\n\n# turn on wandb logging for langchain\nos.environ[\"LANGCHAIN\\_WANDB\\_TRACING\"] = \"true\"\n\n``` \n\nThats it! Now any call to a LangChain LLM, Chain, Tool or Agent will be logged to Weights & Biases.", "matching_document_document_id": "6adad621-230b-42f0-badc-1738fef555a6", "query_weight": 1.2541935483870967 }, { "query_id": "8062c89c-e760-4a57-a02d-cbfd6f116b88", "query": "W&B report sharing", "matching_document": "--- \n\n## description: Collaborate and share W&B Reports with peers, co-workers, and your team. \n\n# Collaborate on reports \n\nCollaborate and Share W&B Reports \n\nOnce you have saved a report, you can select the **Share** button to collaborate. A draft copy of the report is created when you select the **Edit** button. Draft reports auto-save. Select **Save to report** to publish your changes to the shared report. \n\nA warning notification will appear if an edit conflict occurs. This can occur if you and another collaborator edit the same report at the same time. The warning notification will guide you to resolve potential edit conflicts. \n\n### Comment on reports \n\nClick the comment button on a panel in a report to add a comment directly to that panel. \n\n### Who can edit and share reports? \n\nReports that are created within an individual's private project is only visible to that user. The user can share their project to a team or to the public.", "matching_document_document_id": "7fe941de-f8fe-4acc-b460-004c8b6a298e", "query_weight": 0.6038709677419355 }, { "query_id": "f9f1e8b8-1602-40b7-b275-d38ab134c5a7", "query": "What are common issues when logging distributed training with wandb?", "matching_document": "## Train model with DDP\nThe preceding image demonstrates the W&B App UI dashboard. On the sidebar we see two experiments. One labeled 'null' and a second (bound by a yellow box) called 'DPP'. If you expand the group (select the Group dropdown) you will see the W&B Runs that are associated to that experiment. \n\n### Use W&B Service to avoid common distributed training issues. \n\nThere are two common issues you might encounter when using W&B and distributed training: \n\n1. **Hanging at the beginning of training** - A `wandb` process can hang if the `wandb` multiprocessing interferes with the multiprocessing from distributed training.\n2. **Hanging at the end of training** - A training job might hang if the `wandb` process does not know when it needs to exit. Call the `wandb.finish()` API at the end of your Python script to tell W&B that the Run finished. The wandb.finish() API will finish uploading data and will cause W&B to exit. \n\n### Enable W&B Service \n\n### Example use cases for multiprocessing", "matching_document_document_id": "39684e44-a845-4e76-bafe-10b5fda1cf4b", "query_weight": 0.7432258064516128 }, { "query_id": "53de51ff-e1da-415e-b7f4-c69842a579e7", "query": "What are the key features of wandb?", "matching_document": "## Weights & Biases overview\nW&B is a platform that helps data scientists track their models, datasets, system information and more. With a few lines of code, you can start tracking everything about these features. It's free for personal use. Team use is normally a paid utility, but teams for academic purposes are free. You can use W&B with your favourite framework, like TensorFlow, Keras, PyTorch, Sklearn, fastai and many others. \n\nAll tracking information is sent to a dedicated project page on the W&B UI, where you can open high quality visualizations, aggregate information and compare models or parameters. One of the advantages of remotely storing the experiment\u2019s information is that it is easy to collaborate on the same project and share the results with your teammates. \n\nW&B provides 4 useful tools: \n\n* Dashboard: Experiment tracking\n* Artifacts: Dataset versioning, model versioning\n* Sweeps: Hyperparameter optimization\n* Reports: Save and share reproducible findings", "matching_document_document_id": "8c1600ad-9b6e-45cd-86ae-03afd8f80332", "query_weight": 0.9290322580645161 }, { "query_id": "973463b3-b7e5-448a-ad78-924503dee370", "query": "What are the common issues when logging distributed training with wandb?", "matching_document": "## Train model with DDP\nThe preceding image demonstrates the W&B App UI dashboard. On the sidebar we see two experiments. One labeled 'null' and a second (bound by a yellow box) called 'DPP'. If you expand the group (select the Group dropdown) you will see the W&B Runs that are associated to that experiment. \n\n### Use W&B Service to avoid common distributed training issues. \n\nThere are two common issues you might encounter when using W&B and distributed training: \n\n1. **Hanging at the beginning of training** - A `wandb` process can hang if the `wandb` multiprocessing interferes with the multiprocessing from distributed training.\n2. **Hanging at the end of training** - A training job might hang if the `wandb` process does not know when it needs to exit. Call the `wandb.finish()` API at the end of your Python script to tell W&B that the Run finished. The wandb.finish() API will finish uploading data and will cause W&B to exit. \n\n### Enable W&B Service \n\n### Example use cases for multiprocessing", "matching_document_document_id": "39684e44-a845-4e76-bafe-10b5fda1cf4b", "query_weight": 0.7432258064516128 }, { "query_id": "ca5959c1-2b70-407c-a9c7-dda4c3306395", "query": "best practices for tracking experiments in Weights & Biases", "matching_document": "## Create an Experiment\n### Best Practices\n\nThe following are some suggested guidelines to consider when you create experiments: \n\n1. **Config**: Track hyperparameters, architecture, dataset, and anything else you'd like to use to reproduce your model. These will show up in columns\u2014 use config columns to group, sort, and filter runs dynamically in the app.\n2. **Project**: A project is a set of experiments you can compare together. Each project gets a dedicated dashboard page, and you can easily turn on and off different groups of runs to compare different model versions.\n3. **Notes**: A quick commit message to yourself. The note can be set from your script. You can edit notes at a later time on the Overview section of your project's dashboard on the W&B App.\n4. **Tags**: Identify baseline runs and favorite runs. You can filter runs using tags. You can edit tags at a later time on the Overview section of your project's dashboard on the W&B App.", "matching_document_document_id": "fba62bc2-107b-4676-9e4a-ba03137729e2", "query_weight": 0.1 }, { "query_id": "d7a79ed7-1cb9-4a59-8a9f-ce0b0b5ea428", "query": "Wandb init configuration", "matching_document": "## init\n| Arguments | |\n| --- | --- |\n| `project` | (str, optional) The name of the project where you're sending the new run. If the project is not specified, the run is put in an \"Uncategorized\" project. |\n| `entity` | (str, optional) An entity is a username or team name where you're sending runs. This entity must exist before you can send runs there, so make sure to create your account or team in the UI before starting to log runs. If you don't specify an entity, the run will be sent to your default entity, which is usually your username. Change your default entity in your settings under \"default location to create new projects\". |\n| `config` | (dict, argparse, absl.flags, str, optional) This sets `wandb.config`, a dictionary-like object for saving inputs to your job, like hyperparameters for a model or settings for a data preprocessing job. The config will show up in a table in the UI that you can use to group, filter, and sort runs. Keys should not contain `.` in their names, and values should be under 10 MB. If dict, argparse or absl.flags: will load the key value pairs into the `wandb.config` object. If str: will look for a yaml file by that name, and load config from that file into the `wandb.config` object. |\n| `save_code` | (bool, optional) Turn this on to save the main script or notebook to W&B. This is valuable for improving experiment reproducibility and to diff code across experiments in the UI. By default this is off, but you can flip the default behavior to on in your settings page. |\n| `group` | (str, optional) Specify a group to organize individual runs into a larger experiment. For example, you might be doing cross validation, or you might have multiple jobs that train and evaluate a model against different test sets. Group gives you a way to organize runs together into a larger whole, and you can toggle this on and off in the UI. For more details, see our guide to grouping runs. |", "matching_document_document_id": "19b5e055-d931-4a4a-80c4-ec5cd6411109", "query_weight": 0.8825806451612903 }, { "query_id": "ddf99f8f-9e1e-4e21-b4bd-943d24cce1ee", "query": "can you provide a bit more clarity on the difference between setting `resume` in `wandb.init` to `allow` vs. `auto`?\n\nI guess the difference has to do with whether the previous run crashed or not. I guess if the run didn't crash, `auto` may overwrite if there's matching `id`?", "matching_document": "| `resume` | (bool, str, optional) Sets the resuming behavior. Options: `\"allow\"`, `\"must\"`, `\"never\"`, `\"auto\"` or `None`. Defaults to `None`. Cases: - `None` (default): If the new run has the same ID as a previous run, this run overwrites that data. - `\"auto\"` (or `True`): if the previous run on this machine crashed, automatically resume it. Otherwise, start a new run. - `\"allow\"`: if id is set with `init(id=\"UNIQUE_ID\")` or `WANDB_RUN_ID=\"UNIQUE_ID\"` and it is identical to a previous run, wandb will automatically resume the run with that id. Otherwise, wandb will start a new run. - `\"never\"`: if id is set with `init(id=\"UNIQUE_ID\")` or `WANDB_RUN_ID=\"UNIQUE_ID\"` and it is identical to a previous run, wandb will crash. - `\"must\"`: if id is set with `init(id=\"UNIQUE_ID\")` or `WANDB_RUN_ID=\"UNIQUE_ID\"` and it is identical to a previous run, wandb will automatically resume the run with the id. Otherwise, wandb will crash. See our guide to resuming runs for more. |\n| `reinit` | (bool, optional) Allow multiple `wandb.init()` calls in the same process. (default: `False`) |\n| `magic` | (bool, dict, or str, optional) The bool controls whether we try to auto-instrument your script, capturing basic details of your run without you having to add more wandb code. (default: `False`) You can also pass a dict, json string, or yaml filename. |\n| `config_exclude_keys` | (list, optional) string keys to exclude from `wandb.config`. |\n| `config_include_keys` | (list, optional) string keys to include in `wandb.config`. |", "matching_document_document_id": "01c9bdf0-fd26-4350-8de1-e75588077941", "query_weight": 1.0219354838709678 }, { "query_id": "c90d0c2a-dc68-4bac-947d-aaa2b3b19bb1", "query": "W&B code artifacts", "matching_document": "--- \n\n## displayed\\_sidebar: default \n\n# Code Saving \n\nBy default, we only save the latest git commit hash. You can turn on more code features to compare the code between your experiments dynamically in the UI. \n\nStarting with `wandb` version 0.8.28, we can save the code from your main training file where you call `wandb.init()`. This will get sync'd to the dashboard and show up in a tab on the run page, as well as the Code Comparer panel. Go to your settings page to enable code saving by default. \n\n## Save Library Code \n\nWhen code saving is enabled, wandb will save the code from the file that called `wandb.init()`. To save additional library code, you have two options: \n\n* Call `wandb.run.log_code(\".\")` after calling `wandb.init()`\n* Pass a settings object to `wandb.init` with code\\_dir set: `wandb.init(settings=wandb.Settings(code_dir=\".\"))` \n\n## Code Comparer \n\n## Jupyter Session History \n\n## Jupyter diffing", "matching_document_document_id": "10c3ed4d-db30-4f77-b9c9-d90af6894203", "query_weight": 1.3935483870967742 }, { "query_id": "33945412-3dd7-47d0-bf7d-40b6c6a0d088", "query": "How to transfer runs between projects in wandb?", "matching_document": "--- \n\n## displayed\\_sidebar: default \n\n# Manage Runs \n\n### Move runs to a team \n\nOn the project page: \n\n1. Click the table tab to expand the runs table\n2. Click the checkbox to select all runs\n3. Click **Move**: the destination project can be in your personal account or any team that you're a member of. \n\n### Send new runs to a team \n\nIn your script, set the entity to your team. \"Entity\" just means your username or team name. Create an entity (personal account or team account) in the web app before sending runs there. \n\n```\nwandb.init(entity=\"example-team\")\n\n``` \n\nYour **default entity** is updated when you join a team. This means that on your settings page, you'll see that the default location to create a new project is now the team you've just joined. Here's an example of what that settings page section looks like:", "matching_document_document_id": "3e05fcaf-8f88-491a-99cd-2e550d1c9085", "query_weight": 2.136774193548387 }, { "query_id": "e9cdca9f-dc3e-414c-ad96-e53ca57214b3", "query": "Implementing Weights & Biases in a PyTorch project", "matching_document": "## \ud83d\udd25 = W&B \u2795 PyTorch\n\nUse Weights & Biases for machine learning experiment tracking, dataset versioning, and project collaboration. \n\n## What this notebook covers: \n\nWe show you how to integrate Weights & Biases with your PyTorch code to add experiment tracking to your pipeline. \n\n## The resulting interactive W&B dashboard will look like: \n\n## In pseudocode, what we'll do is: \n\n```\n# import the library\nimport wandb\n\n# start a new experiment\nwandb.init(project=\"new-sota-model\")\n\n#\u2003capture a dictionary of hyperparameters with config\nwandb.config = {\"learning\\_rate\": 0.001, \"epochs\": 100, \"batch\\_size\": 128}\n\n# set up model and data\nmodel, dataloader = get\\_model(), get\\_data()\n\n# optional: track gradients\nwandb.watch(model)\n\nfor batch in dataloader:\nmetrics = model.training\\_step()\n#\u2003log metrics inside your training loop to visualize model performance\nwandb.log(metrics)\n\n# optional: save model at the end\nmodel.to\\_onnx()\nwandb.save(\"model.onnx\")\n\n``` \n\n## Follow along with a video tutorial!", "matching_document_document_id": "ee62983f-f238-4b4a-bde8-6b4c068d7918", "query_weight": 1.2541935483870967 }, { "query_id": "85487d87-02e8-4452-8d18-2a240d8a97fd", "query": "confusion matrix visualization", "matching_document": "## Step 2: Create a custom chart for the confusion matrix\nW&B custom charts are written in [Vega](https://vega.github.io/vega/), a powerful and flexible visualization language. You can find many examples and walkthroughs online, and it can help to start with an existing preset that is most similar to your desired custom visualization. You can iterate from small changes in our [IDE](https://wandb.ai/wandb/posts/reports/The-W-B-Machine-Learning-Visualization-IDE--VmlldzoyOTQxNDY5/edit), which renders the plot as you change its definition. \n\nHere is the full Vega spec for this multi-class confusion matrix: \n\n* From your project workspace or report, click on \"Add a visualization\" and select \"Custom chart\"\n* Pick any existing preset and replace its definition with the Vega spec below\n* Click \"Save As\" to give this preset a name for easier reference (I recommend \"confusion_matrix\" :)", "matching_document_document_id": "0c28a653-58b5-42dc-9742-7cb178bdee9b", "query_weight": 0.8361290322580645 }, { "query_id": "7e58b85c-b91e-4d7e-8ee1-ee687a05138a", "query": "Weights & Biases features for LLM developers", "matching_document": "**Weights & Biases Prompts** is a suite of LLMOps tools built for the development of LLM-powered applications. \n\nUse W&B Prompts to visualize and inspect the execution flow of your LLMs, analyze the inputs and outputs of your LLMs, view the intermediate results and securely store and manage your prompts and LLM chain configurations. \n\n#### \ud83e\ude84 View Prompts In Action \n\n**In this notebook we will demostrate W&B Prompts:** \n\n* Using our 1-line LangChain integration\n* Using our Trace class when building your own LLM Pipelines \n\nSee here for the full W&B Prompts documentation \n\n## Installation \n\n```\n!pip install \"wandb>=0.15.4\" -qqq\n!pip install \"langchain>=0.0.218\" openai -qqq\n\n``` \n\n```\nimport langchain\nassert langchain.__version__ >= \"0.0.218\", \"Please ensure you are using LangChain v0.0.188 or higher\"\n\n``` \n\n## Setup \n\nThis demo requires that you have an OpenAI key \n\n# W&B Prompts \n\nW&B Prompts consists of three main components: \n\n**Trace table**: Overview of the inputs and outputs of a chain.", "matching_document_document_id": "5ccb89d6-6353-4b96-9920-060f32aad458", "query_weight": 1.207741935483871 }, { "query_id": "04056895-371a-4a1a-b909-0cb8c39ec668", "query": "tracking artifacts in wandb", "matching_document": "## Artifacts\n### How it works\nCreate an artifact with four lines of code:\n1. Create a W&B run.\n2. Create an artifact object with the `wandb.Artifact` API.\n3. Add one or more files, such as a model file or dataset, to your artifact object.\n4. Log your artifact to W&B. \n\n`python showLineNumbers\nrun = wandb.init(project=\"artifacts-example\", job_type=\"add-dataset\")\nartifact = wandb.Artifact(name=\"my_data\", type=\"dataset\")\nartifact.add_dir(local_path=\"./dataset.h5\") # Add dataset directory to artifact\nrun.log_artifact(artifact) # Logs the artifact version \"my_data:v0\"` \n\n:::tip\nThe preceding code snippet, and the colab linked on this page, show how to track files by uploading them to W&B. See the track external files page for information on how to add references to files or directories that are stored in external object storage (for example, in an Amazon S3 bucket).\n:::", "matching_document_document_id": "01d94aae-eec0-46ba-9972-915e551f3d88", "query_weight": 0.1 }, { "query_id": "603b12ed-f35f-44b1-81bc-d35561185fe2", "query": "Creating and tracking dataset versions using W&B", "matching_document": "## Dataset Versioning\n### Core Artifacts features\n1. **Upload**: Start tracking and versioning any data (files or directories) with `run.log_artifact()`. You can also track datasets in a remote filesystem (e.g. cloud storage in S3 or GCP) by reference, using a link or URI instead of the raw contents.\n2. **Version**: Define an artifact by giving it a type (`\"raw_data\"`, `\"preprocessed_data\"`, `\"balanced_data\"`) and a name (`\"imagenet_cats_10K\"`). When you log the same name again, W&B automatically creates a new version of the artifact with the latest contents.\n3. **Alias**: Set an alias like `\"best\"` or `\"production\"` to highlight the important versions in a lineage of artifacts.\n4. **Compare**: Select any two versions to browse the contents side-by-side. We're also working on a tool for dataset visualization, learn more here.\n5. **Download**: Obtain a local copy of the artifact or verify the contents by reference. \n\nFor more detail on these features, check out Artifacts how it works.", "matching_document_document_id": "7407e418-a665-438f-b605-d5dc12e4240e", "query_weight": 0.1 }, { "query_id": "24e0d7e2-6ef6-4b6f-a3f8-dd6d4d0f4eb1", "query": "wandb.Table documentation", "matching_document": "## Table\n\nView source on GitHub \n\nThe Table class used to display and analyze tabular data. \n\n```\nTable(\ncolumns=None, data=None, rows=None, dataframe=None, dtype=None, optional=(True),\nallow\\_mixed\\_types=(False)\n)\n\n``` \n\nUnlike traditional spreadsheets, Tables support numerous types of data:\nscalar values, strings, numpy arrays, and most subclasses of `wandb.data_types.Media`.\nThis means you can embed `Images`, `Video`, `Audio`, and other sorts of rich, annotated media\ndirectly in Tables, alongside other traditional scalar values. \n\nThis class is the primary class used to generate the Table Visualizer\nin the UI: . \n\nTables can be constructed with initial data using the `data` or\n`dataframe` parameters:", "matching_document_document_id": "ef293fdf-142d-4a6e-b668-514932593e8f", "query_weight": 0.1 }, { "query_id": "b37bc67b-b992-4445-b178-1a47956e7af2", "query": "Weights & Biases artifact storage", "matching_document": "## Storage\n\nIf you are approaching or exceeding your storage limit, there are multiple paths forward to manage your data. The path that's best for you will depend on your account type and your current project setup. \n\n## Manage storage consumption \n\nW&B offers different methods of optimizing your storage consumption: \n\n* Use\u00a0reference artifacts\u00a0to track files saved outside the W&B system, instead of uploading them to W&B storage.\n* Use an external cloud storage bucket for storage. *(Enterprise only)* \n\n## Delete data \n\nYou can also choose to delete data to remain under your storage limit. There are several ways to do this: \n\n* Delete data interactively with the app UI.\n* Set a TTL policy on Artifacts so they are automatically deleted.", "matching_document_document_id": "6dea2d7d-58f2-4830-8dc1-27b3bf719859", "query_weight": 0.5109677419354839 }, { "query_id": "96b946ef-fc24-4e6e-8215-47cfb00f015e", "query": "W&B sweep troubleshooting", "matching_document": "## Troubleshoot Sweeps\n\nTroubleshoot W&B Sweeps \n\nTroubleshoot common error messages with the guidance suggested. \n\n### `CommError, Run does not exist` and `ERROR Error uploading` \n\nYour W&B Run ID might be defined if these two error messages are both returned. As an example, you might have a similar code snippet defined somewhere in your Jupyter Notebooks or Python script: \n\n```\nwandb.init(id=\"some-string\")\n\n``` \n\nYou can not set a Run ID for W&B Sweeps because W&B automatically generates random, unique IDs for Runs created by W&B Sweeps. \n\nW&B Run IDs need to be unique within a project. \n\nWe recommend you pass a name to the name parameter when you initialized W&B, if you want to set a custom name that will appear on tables and graphs. For example: \n\n```\nwandb.init(name=\"a helpful readable run name\")\n\n``` \n\n### `Cuda out of memory` \n\n### `anaconda 400 error` \n\nThe following error usually occurs when you do not log the metric that you are optimizing:", "matching_document_document_id": "13152c97-59c9-456d-b955-55a590204565", "query_weight": 0.1 }, { "query_id": "78f710df-0a7e-46c1-b134-db27395d8737", "query": "W&B team collaboration features", "matching_document": "## Teams\n\nUse W&B Teams as a central workspace for your ML team to build better models faster. \n\n* **Track all the experiments** your team has tried so you never duplicate work.\n* **Save and reproduce** previously trained models.\n* **Share progress** and results with your boss and collaborators.\n* **Catch regressions** and immediately get alerted when performance drops.\n* **Benchmark model performance** and compare model versions. \n\n## Create a collaborative team \n\n1. **Sign up or log in** to your free W&B account.\n2. Click **Invite Team** in the navigation bar.\n3. Create your team and invite collaborators. \n\n:::info\n**Note**: Only the admin of an organization can create a new team.\n::: \n\n## Create a Team Profile \n\nYou can customize your team's profile page to show an introduction and showcase reports and projects that are visible to the public or team members. Present reports, projects, and external links. \n\n## Remove Team members \n\n## Team Roles and Permissions \n\n## Team Trials", "matching_document_document_id": "878a2967-599f-4092-9854-920a07a186ba", "query_weight": 0.1 }, { "query_id": "e6a928f6-fb9c-4a73-9b65-752ca0990382", "query": "Weights & Biases artifact API", "matching_document": "Arguments:\n name: A human-readable name for the artifact. Use the name to identify\n a specific artifact in the W&B App UI or programmatically. You can\n interactively reference an artifact with the `use_artifact` Public API.\n A name can contain letters, numbers, underscores, hyphens, and dots.\n The name must be unique across a project.\n type: The artifact's type. Use the type of an artifact to both organize\n and differentiate artifacts. You can use any string that contains letters,\n numbers, underscores, hyphens, and dots. Common types include `dataset` or `model`.\n Include `model` within your type string if you want to link the artifact\n to the W&B Model Registry.\n description: A description of the artifact. For Model or Dataset Artifacts,\n add documentation for your standardized team model or dataset card. View\n an artifact's description programmatically with the `Artifact.description`\n attribute or programmatically with the W&B App UI. W&B renders the\n description as markdown in the W&B App.\n metadata: Additional information about an artifact. Specify metadata as a\n dictionary of key-value pairs. You can specify no more than 100 total keys.\n\n Returns:\n An `Artifact` object.\n \"\"\"", "matching_document_document_id": "be3722cf-110a-46ac-a0e1-58aba5de5745", "query_weight": 0.1 }, { "query_id": "35860a55-3f14-4de9-98a4-e735e7f092db", "query": "how to define W&B sweep in YAML", "matching_document": "## Add W&B to your code\n#### Training script with W&B Python SDK\nTo create a W&B Sweep, we first create a YAML configuration file. The configuration file contains he hyperparameters we want the sweep to explore. In the proceeding example, the batch size (`batch_size`), epochs (`epochs`), and the learning rate (`lr`) hyperparameters are varied during each sweep. \n\n```\n# config.yaml\nprogram: train.py\nmethod: random\nname: sweep\nmetric:\ngoal: maximize\nname: val\\_acc\nparameters:\nbatch\\_size:\nvalues: [16,32,64]\nlr:\nmin: 0.0001\nmax: 0.1\nepochs:\nvalues: [5, 10, 15]\n\n``` \n\nFor more information on how to create a W&B Sweep configuration, see Define sweep configuration. \n\nNote that you must provide the name of your Python script for the `program` key in your YAML file. \n\nNext, we add the following to the code example:", "matching_document_document_id": "77cd3e2d-cd4c-4d88-8ffb-2bc94492b051", "query_weight": 0.7896774193548387 }, { "query_id": "5dc2c71c-00c2-4be4-823a-05347d8154c0", "query": "Version control of datasets in machine learning projects", "matching_document": "## Version Control in Machine Learning\n### Data Version Control\n* Data preprocessing (data content changes) such as data cleaning, outlier handling, filling of missing values, etc.\n* Feature engineering (data becomes \"wider\") such as aggregation features, label encoding, scaling, etc.\n* Dataset splits (data is partitioned) typically mean dividing your data into training, validation, and testing data.\n* Dataset update (data becomes \"longer\") when new data points are available.", "matching_document_document_id": "152cb478-f767-4f0f-b64b-7a86691a728b", "query_weight": 1.904516129032258 }, { "query_id": "54d7cff9-416c-4312-a458-a59908b4e306", "query": "Weights & Biases model registry tutorial", "matching_document": "## Model registry\n### How it works\n1. **Log a model version**: In your training script, add a few lines of code to save the model files as an artifact to W&B.\n2. **Compare performance**: Check live charts to compare the metrics and sample predictions from model training and validation. Identify which model version performed the best.\n3. **Link to registry**: Bookmark the best model version by linking it to a registered model, either programmatically in Python or interactively in the W&B UI. \n\nThe following code snippet demonstrates how to log and link a model to the Model Registry: \n\n```python showLineNumbers\nimport wandb\nimport random\n\n# Start a new W&B run\n\nrun = wandb.init(project=\"models\\_quickstart\")\n\n# Simulate logging model metrics\n\nrun.log({\"acc\": random.random()})\n\n# Create a simulated model file\n\nwith open(\"my\\_model.h5\", \"w\") as f:\nf.write(\"Model: \" + str(random.random()))\n\n# Log and link the model to the Model Registry\n\nrun.link\\_model(path=\"./my\\_model.h5\", registered\\_model\\_name=\"MNIST\")\n\nrun.finish()\n```", "matching_document_document_id": "c8056eb1-14f6-47e4-82a9-b839839eb916", "query_weight": 0.1 }, { "query_id": "29106b19-3e73-4fc7-9f03-f0182753bd45", "query": "access artifact path W&B", "matching_document": "## Artifact\n### Methods\n#### `get_added_local_path_name`\n\nView source \n\n```\nget\\_added\\_local\\_path\\_name(\nlocal\\_path: str\n) -> Optional[str]\n\n``` \n\nGet the artifact relative name of a file added by a local filesystem path. \n\n| Arguments | |\n| --- | --- |\n| `local_path` | The local path to resolve into an artifact relative name. | \n\n| Returns | |\n| --- | --- |\n| The artifact relative name. | | \n\n#### Examples: \n\nBasic usage: \n\n```\nartifact = wandb.Artifact(\"my\\_dataset\", type=\"dataset\")\nartifact.add\\_file(\"path/to/file.txt\", name=\"artifact/path/file.txt\")\n\n# Returns `artifact/path/file.txt`:\nname = artifact.get\\_added\\_local\\_path\\_name(\"path/to/file.txt\")\n\n```", "matching_document_document_id": "9e0404bc-42ab-435c-80b4-d5f5b1d553e3", "query_weight": 1.3935483870967742 }, { "query_id": "7bbab76d-aa7a-4d9b-b88d-3103716d4d62", "query": "optimize W&B storage", "matching_document": "## Storage\n\nIf you are approaching or exceeding your storage limit, there are multiple paths forward to manage your data. The path that's best for you will depend on your account type and your current project setup. \n\n## Manage storage consumption \n\nW&B offers different methods of optimizing your storage consumption: \n\n* Use\u00a0reference artifacts\u00a0to track files saved outside the W&B system, instead of uploading them to W&B storage.\n* Use an external cloud storage bucket for storage. *(Enterprise only)* \n\n## Delete data \n\nYou can also choose to delete data to remain under your storage limit. There are several ways to do this: \n\n* Delete data interactively with the app UI.\n* Set a TTL policy on Artifacts so they are automatically deleted.", "matching_document_document_id": "6dea2d7d-58f2-4830-8dc1-27b3bf719859", "query_weight": 0.1 }, { "query_id": "5f9e6e76-bcb9-48b5-9337-8f4504395b95", "query": "What does setting the 'resume' parameter to 'allow' do in wandb.init?", "matching_document": "## Resume Runs\n#### Resume Guidance\n##### Automatic and controlled resuming\n\nAutomatic resuming only works if the process is restarted on top of the same filesystem as the failed process. If you can't share a filesystem, we allow you to set the `WANDB_RUN_ID`: a globally unique string (per project) corresponding to a single run of your script. It must be no longer than 64 characters. All non-word characters will be converted to dashes. \n\n```\n# store this id to use it later when resuming\nid = wandb.util.generate\\_id()\nwandb.init(id=id, resume=\"allow\")\n# or via environment variables\nos.environ[\"WANDB\\_RESUME\"] = \"allow\"\nos.environ[\"WANDB\\_RUN\\_ID\"] = wandb.util.generate\\_id()\nwandb.init()\n\n``` \n\nIf you set `WANDB_RESUME` equal to `\"allow\"`, you can always set `WANDB_RUN_ID` to a unique string and restarts of the process will be handled automatically. If you set `WANDB_RESUME` equal to `\"must\"`, W&B will throw an error if the run to be resumed does not exist yet instead of auto-creating a new run.", "matching_document_document_id": "717baee4-2e53-4b5c-b345-41f09c0ff195", "query_weight": 1.0219354838709678 }, { "query_id": "9329ba15-8755-429b-8c82-d95805f59ace", "query": "wandb.log confusion matrix chart", "matching_document": "' \n\n# Confusion Matrix: Usage and Examples \n\nDescription: In this article, we review the usage and examples for a multi-class confusion matrix using Weights & Biases. \n\nBody: \n\n# Method: wandb.plot.confusion_matrix() \n\nLog a multi-class [confusion matrix](https://scikit-learn.org/stable/auto_examples/model_selection/plot_confusion_matrix.html) in one line: \n\n```\nwandb.log({\"conf_mat\" : wandb.plot.confusion_matrix(probs=None,\ny_true=ground_truth, preds=predictions,\nclass_names=class_names)})\n``` \n\nYou can log this wherever your code has access to: \n\n# Basic Usage \n\nIn this toy example, I finetune a CNN to predict one of 10 classes of living things in a photo (plants, animals, insects) while varying the number of epochs (E or pretrain_epochs) and the number of training examples (NT or num_train). I log a confusion matrix in each validation step, after each training epoch (so only the last confusion matrix at the end of training is visualized). \n\n# Powerful Interactions \n\n# Logging Details", "matching_document_document_id": "f3700051-8c7a-48b4-bf96-0e5fef0c6039", "query_weight": 0.8361290322580645 }, { "query_id": "8e3be407-2e9c-4f39-ad37-227906befc59", "query": "using LangChain with Weights & Biases", "matching_document": "## Prompts Quickstart\n### Use W&B Trace with LangChain\n#### 1. Set the LANGCHAIN\\_WANDB\\_TRACING environment variable\nFirst, set the LANGCHAIN\\_WANDB\\_TRACING environment variable to true. This will turn on automated Weights & Biases logging with LangChain: \n\n```\nimport os\n\n# turn on wandb logging for langchain\nos.environ[\"LANGCHAIN\\_WANDB\\_TRACING\"] = \"true\"\n\n``` \n\nThats it! Now any call to a LangChain LLM, Chain, Tool or Agent will be logged to Weights & Biases.", "matching_document_document_id": "6adad621-230b-42f0-badc-1738fef555a6", "query_weight": 1.2541935483870967 }, { "query_id": "ff93dae5-ead7-4d27-a3ab-a4a376663963", "query": "WandB API examples for sweeps", "matching_document": "\"\"\"A set of runs associated with a sweep.\n\n Examples:\n Instantiate with:\n ```\n api = wandb.Api()\n sweep = api.sweep(path/to/sweep)\n ```\n\n Attributes:\n runs: (`Runs`) list of runs\n id: (str) sweep id\n project: (str) name of project\n config: (str) dictionary of sweep configuration\n state: (str) the state of the sweep\n expected_run_count: (int) number of expected runs for the sweep\n \"\"\"", "matching_document_document_id": "71688f09-0d8d-48a1-a6e5-8e33a63d6f43", "query_weight": 0.6503225806451612 }, { "query_id": "e961a5f9-2447-4925-ae7e-8f34e8418a02", "query": "What are the steps to connect LangChain with Weights & Biases?", "matching_document": "## Prompts Quickstart\n### Use W&B Trace with LangChain\n#### 1. Set the LANGCHAIN\\_WANDB\\_TRACING environment variable\nFirst, set the LANGCHAIN\\_WANDB\\_TRACING environment variable to true. This will turn on automated Weights & Biases logging with LangChain: \n\n```\nimport os\n\n# turn on wandb logging for langchain\nos.environ[\"LANGCHAIN\\_WANDB\\_TRACING\"] = \"true\"\n\n``` \n\nThats it! Now any call to a LangChain LLM, Chain, Tool or Agent will be logged to Weights & Biases.", "matching_document_document_id": "6adad621-230b-42f0-badc-1738fef555a6", "query_weight": 1.2541935483870967 }, { "query_id": "1a123420-2fcc-4671-aacc-7db790f9968b", "query": "How to version models using Weights & Biases model registry?", "matching_document": "## Link a model version\n### Interactively link a model\n1. Navigate to your project's artifact browser on the W&B App at: `https://wandb.ai///artifacts`\n2. Select the Artifacts icon on the left sidebar.\n3. Click on the model version you want to link to your registry.\n4. Within the **Version overview** section, click the **Link to registry** button.\n5. From the modal that appears on the right of the screen, select a registered model from the **Select a register model** menu dropdown.\n6. Click **Next step**.\n7. (Optional) Select an alias from the **Aliases** dropdown.\n8. Click **Link to registry**.", "matching_document_document_id": "d7b224e1-1c73-4711-94e3-f4330e0f912c", "query_weight": 1.0219354838709678 }, { "query_id": "bde88efe-6718-4faf-870c-b2139de5daf5", "query": "How to invite team members to a shared project in Weights & Biases?", "matching_document": "## Teams\n### Invite team members\nInvite new members to your team.\n1. Ensure the team member already has a W&B Account.\n2. Navigate to .\n3. Select **Manage members**.\n4. A model will appear. Provide the username or email for the **Email or Username** field, select a team for them to join from the **Team** dropdown menu, and select a role type from the **Organizational Role** dropdown menu. \n\n1. Select the **Add** button. \n\n:::info\n\\* If you have an Enterprise account, please contact your Account Executive to invite new members to your team.\n:::", "matching_document_document_id": "5c1c7d28-53bf-45fb-89c2-1219c423b66f", "query_weight": 2.136774193548387 }, { "query_id": "54330a7f-91ce-489b-aee6-df46f172655d", "query": "Weights & Biases reports", "matching_document": "## Create a report\n\nCreate a W&B Report \n\nCreate a report interactively with the W&B App UI or programmatically with the W&B Python SDK. \n\n:::info\nCreating Reports programmatically with the Python SDK is in Beta and in active development. See this Google Colab for an example.\n::: \n\n1. Navigate to your project workspace in the W&B App.\n2. Click **Create report** in the upper right corner of your workspace. \n\n1. A modal will appear. Select the charts you would like to start with. You can add or delete charts later from the report interface. \n\n1. Select the **Filter run sets** option to prevent new runs from being added to your report. You can toggle this option on or off. Once you click **Create report,** a draft report will be available in the report tab to continue working on. \n\n1. Navigate to your project workspace in the W&B App.\n2. Select to the **Reports** tab (clipboard image) in your project.\n3. Select the **Create Report** button on the report page.", "matching_document_document_id": "a27b93de-1c23-4644-9d16-9a40c305a091", "query_weight": 0.9754838709677419 }, { "query_id": "d5fec68e-cdc4-44ed-89a7-b7a9d1487f25", "query": "wandb setup", "matching_document": "## \ud83d\ude80 Setup\n\nStart out by installing the experiment tracking library and setting up your free W&B account: \n\n1. Install with `!pip install`\n2. `import` the library into Python\n3. `.login()` so you can log metrics to your projects \n\nIf you've never used Weights & Biases before,\nthe call to `login` will give you a link to sign up for an account.\nW&B is free to use for personal and academic projects! \n\n```\n!pip install wandb -Uq\n\n``` \n\n```\nimport wandb\n\n``` \n\n```\nwandb.login()\n\n```", "matching_document_document_id": "ee82f3a4-c71a-434e-a6fe-0170efcb2674", "query_weight": 1.1148387096774193 }, { "query_id": "68b8f14f-702d-43ba-b7c4-72a0f382a316", "query": "Examples of logging confusion matrix as a chart in Weights & Biases", "matching_document": "' \n\n# Confusion Matrix: Usage and Examples \n\nDescription: In this article, we review the usage and examples for a multi-class confusion matrix using Weights & Biases. \n\nBody: \n\n# Method: wandb.plot.confusion_matrix() \n\nLog a multi-class [confusion matrix](https://scikit-learn.org/stable/auto_examples/model_selection/plot_confusion_matrix.html) in one line: \n\n```\nwandb.log({\"conf_mat\" : wandb.plot.confusion_matrix(probs=None,\ny_true=ground_truth, preds=predictions,\nclass_names=class_names)})\n``` \n\nYou can log this wherever your code has access to: \n\n# Basic Usage \n\nIn this toy example, I finetune a CNN to predict one of 10 classes of living things in a photo (plants, animals, insects) while varying the number of epochs (E or pretrain_epochs) and the number of training examples (NT or num_train). I log a confusion matrix in each validation step, after each training epoch (so only the last confusion matrix at the end of training is visualized). \n\n# Powerful Interactions \n\n# Logging Details", "matching_document_document_id": "f3700051-8c7a-48b4-bf96-0e5fef0c6039", "query_weight": 0.1 }, { "query_id": "53356b44-7932-4312-a022-88b20438f234", "query": "W&B features for LLM applications", "matching_document": "\" \n\n# Introducing W&B Prompts \n\nDescription: Learn all about W&B's new LLMOps tools and how to use them \n\nBody: \n\n# Introduction \n\nWe are very excited to announce the release of [W&B Prompts,](https://docs.wandb.ai/guides/prompts) a new suite of LLMOps tools. You can try Prompts [here](https://colab.research.google.com/github/wandb/examples/blob/master/colabs/prompts/W%26B_Prompts_Quickstart.ipynb). \n\nWe\u2019re also delighted to announce other features to support the new prompt engineering use case: \n\n* A one-line LangChain integration for effortless logging of LangChain models, inputs and outputs\n* A one-line OpenAI integration to log OpenAI model inputs and outputs\n* A for quick and frictionless evaluation\n* Improved handling of text in W&B Tables including markdown rendering, long text scrolling and string diffs\n* A new W&B JavaScript SDK to track prompt exploration as well as model generations in production \n\nFinally, to contribute to and grow the LLM ecosystem we\u2019re also: \n\n# W&B JavaScript SDK", "matching_document_document_id": "29f9b25f-5e62-496a-8cdf-0a739659b3d1", "query_weight": 0.1 }, { "query_id": "d4ee8921-4d65-4583-910f-c583ea9dd061", "query": "How to structure wandb runs for easy comparison and tracking?", "matching_document": "## Basic Setup\n\n1. **Projects**: Log multiple runs to a project to compare them. `wandb.init(project=\"project-name\")`\n2. **Groups**: For multiple processes or cross validation folds, log each process as a runs and group them together. `wandb.init(group='experiment-1')`\n3. **Tags**: Add tags to track your current baseline or production model.\n4. **Notes**: Type notes in the table to track the changes between runs.\n5. **Reports**: Take quick notes on progress to share with colleagues and make dashboards and snapshots of your ML projects.", "matching_document_document_id": "a3087cfb-91f1-408c-8347-e228b9457651", "query_weight": 0.1 }, { "query_id": "cb7b1739-8d52-43bb-abe1-5d839eaf4548", "query": "How to version datasets in Weights & Biases?", "matching_document": "## Dataset Versioning\n### Core Artifacts features\n1. **Upload**: Start tracking and versioning any data (files or directories) with `run.log_artifact()`. You can also track datasets in a remote filesystem (e.g. cloud storage in S3 or GCP) by reference, using a link or URI instead of the raw contents.\n2. **Version**: Define an artifact by giving it a type (`\"raw_data\"`, `\"preprocessed_data\"`, `\"balanced_data\"`) and a name (`\"imagenet_cats_10K\"`). When you log the same name again, W&B automatically creates a new version of the artifact with the latest contents.\n3. **Alias**: Set an alias like `\"best\"` or `\"production\"` to highlight the important versions in a lineage of artifacts.\n4. **Compare**: Select any two versions to browse the contents side-by-side. We're also working on a tool for dataset visualization, learn more here.\n5. **Download**: Obtain a local copy of the artifact or verify the contents by reference. \n\nFor more detail on these features, check out Artifacts how it works.", "matching_document_document_id": "7407e418-a665-438f-b605-d5dc12e4240e", "query_weight": 1.3006451612903225 }, { "query_id": "4513d724-63fd-4ea3-ab2c-dde774569c40", "query": "monitoring LLM-powered applications", "matching_document": "## Prompts for LLMs\n\nW&B Prompts is a suite of LLMOps tools built for the development of LLM-powered applications. Use W&B Prompts to visualize and inspect the execution flow of your LLMs, analyze the inputs and outputs of your LLMs, view the intermediate results and securely store and manage your prompts and LLM chain configurations. \n\n## Use Cases \n\nW&B Prompts provides several solutions for building and monitoring LLM-based apps. Software developers, prompt engineers, ML practitioners, data scientists, and other stakeholders working with LLMs need cutting-edge tools to: \n\n* Explore and debug LLM chains\u00a0and prompts with greater granularity.\n* Monitor and observe LLMs to better understand and evaluate performance, usage, and budgets. \n\n## Products \n\n### Traces \n\nW&B\u2019s LLM tool is called\u00a0*Traces*.\u00a0**Traces**\u00a0allow you to track and visualize the inputs and outputs, execution flow, model architecture, and any intermediate results of your LLM chains. \n\n### Weave \n\n### How it works \n\n## Integrations", "matching_document_document_id": "99fd9bc0-a4ff-4291-9e1c-c6aab2b9b31a", "query_weight": 1.207741935483871 }, { "query_id": "ca03cead-3d24-40e6-afe1-8e9e6527b7fc", "query": "Integrating Weights & Biases with AWS SageMaker for model monitoring.", "matching_document": "## Using AWS Sagemaker and Weights & Biases Together on Digit Recognition with MNIST\n### Integrating Weights & Biases to AWS SageMaker\n2) Pass Weights & Biases API Key to the Estimator \n\nGo to [https://wandb.ai](https://wandb.ai/) to signup for a free account, after which you could go to [https://wandb.ai/authorize](https://wandb.ai/authorize) to get your API key! \n\nYou should see something like the screenshot above. Simply copy the API key and paste it in the following code block. \n\n```\nestimator = PyTorch(entry_point='mnist.py',\nsource_dir=\"src\",\nrole=role,\npy_version='py3',\nframework_version='1.8.0',\ninstance_count=1,\ninstance_type='ml.c5.2xlarge',\nhyperparameters={\n'epochs': 1,\n'backend': 'gloo'\n},\n# Pass Weights & Biases API Key as an environment variable\n# You can find the key by going to https://wandb.ai/authorize\nenvironment={\"WANDB_API_KEY\": current_api_key})\n``` \n\nFirst, run the following to train the estimator: \n\n```\nestimator.fit({'training': inputs})\n``` \n\nThen, you'll see your W&B run in your dashboard. From here, you can check out a variety of metrics such as losses, system metrics, and gradient histograms:", "matching_document_document_id": "1ce62199-dbfc-4dc2-98f0-32ac40659f17", "query_weight": 0.6503225806451612 }, { "query_id": "b970a814-cea6-4770-a4eb-47083dae5cb2", "query": "What are the best practices for versioning datasets in W&B?", "matching_document": "--- \n\n## description: Guide to using Artifacts for dataset versioning \n\n# Dataset Versioning \n\nVersion datasets with W&B Artifacts. \n\nW&B Artifacts help you save and organize machine learning datasets throughout a project's lifecycle. \n\n### Common use cases \n\n1. **Version data seamlessly**, without interrupting your workflow\n2. **Prepackage data splits**, like training, validation, and test sets\n3. **Iteratively refine datasets**, without desynchronizing the team\n4. **Juggle multiple datasets**, as in fine-tuning and domain adaptation\n5. **Visualize & share your data workflow**, keeping all your work in one place \n\n### Flexible tracking and hosting \n\nBeyond these common scenarios, you can use core Artifact features to upload, version, alias, compare, and download data, supporting any custom dataset workflow on local or remote file systems, via S3, GCP, or https. \n\n## Core Artifacts features \n\nW&B Artifacts support dataset versioning through these basic features: \n\n## Version data seamlessly", "matching_document_document_id": "d8093732-c7b0-4442-887b-f1d4c1b56116", "query_weight": 0.1 }, { "query_id": "895e2d6a-01a1-4713-a75c-cf400d18f2d7", "query": "wandb.plot.confusion_matrix", "matching_document": "' \n\n# Confusion Matrix: Usage and Examples \n\nDescription: In this article, we review the usage and examples for a multi-class confusion matrix using Weights & Biases. \n\nBody: \n\n# Method: wandb.plot.confusion_matrix() \n\nLog a multi-class [confusion matrix](https://scikit-learn.org/stable/auto_examples/model_selection/plot_confusion_matrix.html) in one line: \n\n```\nwandb.log({\"conf_mat\" : wandb.plot.confusion_matrix(probs=None,\ny_true=ground_truth, preds=predictions,\nclass_names=class_names)})\n``` \n\nYou can log this wherever your code has access to: \n\n# Basic Usage \n\nIn this toy example, I finetune a CNN to predict one of 10 classes of living things in a photo (plants, animals, insects) while varying the number of epochs (E or pretrain_epochs) and the number of training examples (NT or num_train). I log a confusion matrix in each validation step, after each training epoch (so only the last confusion matrix at the end of training is visualized). \n\n# Powerful Interactions \n\n# Logging Details", "matching_document_document_id": "f3700051-8c7a-48b4-bf96-0e5fef0c6039", "query_weight": 0.8361290322580645 }, { "query_id": "13907e9d-6a39-4f5f-93c5-33676c4ab467", "query": "Dataset versioning in Weights & Biases", "matching_document": "## Dataset Versioning\n### Core Artifacts features\n1. **Upload**: Start tracking and versioning any data (files or directories) with `run.log_artifact()`. You can also track datasets in a remote filesystem (e.g. cloud storage in S3 or GCP) by reference, using a link or URI instead of the raw contents.\n2. **Version**: Define an artifact by giving it a type (`\"raw_data\"`, `\"preprocessed_data\"`, `\"balanced_data\"`) and a name (`\"imagenet_cats_10K\"`). When you log the same name again, W&B automatically creates a new version of the artifact with the latest contents.\n3. **Alias**: Set an alias like `\"best\"` or `\"production\"` to highlight the important versions in a lineage of artifacts.\n4. **Compare**: Select any two versions to browse the contents side-by-side. We're also working on a tool for dataset visualization, learn more here.\n5. **Download**: Obtain a local copy of the artifact or verify the contents by reference. \n\nFor more detail on these features, check out Artifacts how it works.", "matching_document_document_id": "7407e418-a665-438f-b605-d5dc12e4240e", "query_weight": 1.3006451612903225 }, { "query_id": "040777c2-6298-4acf-b00b-74a217c3a124", "query": "using wandb artifacts", "matching_document": "## Tutorial\n### Artifacts\nIn addition to Sweeps and the Dashboard, W&B also provides a useful utility called Artifacts that allows you to log your data and models. In this context, Artifacts are produced objects - the outputs of processes - datasets and models. We'll show you how to use Artifacts, using dataset logging from our previous example. \n\nLet's first load a raw dataset, then create a new Artifact. \n\n```\ndef load_and_log():\nwith wandb.init(project=project_name, job_type=\"load-data\") as run:\n\ndatasets = load_data()\nnames = [\"training\", \"validation\", \"test\"]\n\n# Artifact\nraw_data = wandb.Artifact(\n\"mnist-raw\", type=\"dataset\",\ndescription=\"Raw MNIST dataset, splitted\",\nmetadata={\"source\": \"keras.datasets.mnist\",\n\"train_data\": len(datasets[0].x),\n\"valid_data\": len(datasets[1].x),\n\"test_daata\": len(datasets[2].x)})\n\nfor name, data in zip(names, datasets):\n# Save our datasets\nwith raw_data.new_file(name + \".npz\", mode=\"wb\") as file:\nnp.savez(file, x=data.x, y=data.y)\n#save Artifact\nrun.log_artifact(raw_data)\n\nload_and_log()\n```", "matching_document_document_id": "7aa75be7-3f42-4bec-927d-0561a1869e46", "query_weight": 0.9290322580645161 }, { "query_id": "1ac3e1c7-3acb-4c1c-8c1b-e33544284f15", "query": "wandb parallel training logging", "matching_document": "In distributed training, you can either create a single run in the rank 0 process\n and then log information only from that process, or you can create a run in each process,\n logging from each separately, and group the results together with the `group` argument\n to `wandb.init`. For more details on distributed training with W&B, check out\n [our guide](https://docs.wandb.ai/guides/track/log/distributed-training).\n\n Currently, there is a parallel `Run` object in the `wandb.Api`. Eventually these\n two objects will be merged.\n\n Attributes:\n summary: (Summary) Single values set for each `wandb.log()` key. By\n default, summary is set to the last value logged. You can manually\n set summary to the best value, like max accuracy, instead of the\n final value.\n \"\"\"", "matching_document_document_id": "6fcdf0c3-8da7-4cbb-9a9b-3e958859f1c5", "query_weight": 0.7432258064516128 }, { "query_id": "793da882-8a5b-4d93-bb6f-08cbaa1472bf", "query": "tutorial on using Weights & Biases API for chart data", "matching_document": "--- \n\n## description: Tutorial of using the custom charts feature in the Weights & Biases UI \n\n# Custom Charts Walkthrough \n\nTo go beyond the built-in charts in Weights & Biases, use the new **Custom Charts** feature to control the details of exactly what data you're loading in to a panel and how you visualize that data. \n\n**Overview** \n\n1. Log data to W&B\n2. Create a query\n3. Customize the chart \n\n## 1. Log data to W&B \n\nFirst, log data in your script. Use wandb.config for single points set at the beginning of training, like hyperparameters. Use wandb.log() for multiple points over time, and log custom 2D arrays with wandb.Table(). We recommend logging up to 10,000 data points per logged key. \n\n```\n# Logging a custom table of data\nmy\\_custom\\_data = [[x1, y1, z1], [x2, y2, z2]]\nwandb.log({\"custom\\_data\\_table\": wandb.Table(data=my\\_custom\\_data,\ncolumns = [\"x\", \"y\", \"z\"])})\n\n``` \n\n## 2. Create a query \n\n## 3. Customize the chart \n\n## Bonus: Composite Histograms", "matching_document_document_id": "8f575f23-5d22-4f79-8d95-0096e0c1d573", "query_weight": 1.8116129032258064 }, { "query_id": "6d704129-c3e0-463a-b33b-8ac0068424ef", "query": "What is the structure of a sweep configuration in YAML or JSON format?", "matching_document": "## Sweep configuration structure\n### Basic structure\ntrain.py\n```\nsweep\\_configuration = {\n\"name\": \"sweepdemo\",\n\"method\": \"bayes\",\n\"metric\": {\"goal\": \"minimize\", \"name\": \"validation\\_loss\"},\n\"parameters\": {\n\"learning\\_rate\": {\"min\": 0.0001, \"max\": 0.1},\n\"batch\\_size\": {\"values\": [16, 32, 64]},\n\"epochs\": {\"values\": [5, 10, 15]},\n\"optimizer\": {\"values\": [\"adam\", \"sgd\"]},\n},\n}\n\n``` \n\nDefine a sweep configuration in a YAML file if you want to manage sweeps interactively from the command line (CLI) \n\nconfig.yaml\n```\nprogram: train.py\nname: sweepdemo\nmethod: bayes\nmetric:\ngoal: minimize\nname: validation\\_loss\nparameters:\nlearning\\_rate:\nmin: 0.0001\nmax: 0.1\nbatch\\_size:\nvalues: [16, 32, 64]\nepochs:\nvalues: [5, 10, 15]\noptimizer:\nvalues: [\"adam\", \"sgd\"]\n\n```", "matching_document_document_id": "2a1ce862-7280-49c1-80c4-e4dfdd16914e", "query_weight": 0.7896774193548387 }, { "query_id": "7dab37f4-b54a-447a-b45d-cb53a9880193", "query": "W&B dataset version management", "matching_document": "## Dataset Versioning\n### Core Artifacts features\n1. **Upload**: Start tracking and versioning any data (files or directories) with `run.log_artifact()`. You can also track datasets in a remote filesystem (e.g. cloud storage in S3 or GCP) by reference, using a link or URI instead of the raw contents.\n2. **Version**: Define an artifact by giving it a type (`\"raw_data\"`, `\"preprocessed_data\"`, `\"balanced_data\"`) and a name (`\"imagenet_cats_10K\"`). When you log the same name again, W&B automatically creates a new version of the artifact with the latest contents.\n3. **Alias**: Set an alias like `\"best\"` or `\"production\"` to highlight the important versions in a lineage of artifacts.\n4. **Compare**: Select any two versions to browse the contents side-by-side. We're also working on a tool for dataset visualization, learn more here.\n5. **Download**: Obtain a local copy of the artifact or verify the contents by reference. \n\nFor more detail on these features, check out Artifacts how it works.", "matching_document_document_id": "7407e418-a665-438f-b605-d5dc12e4240e", "query_weight": 0.1 }, { "query_id": "c746d301-bfea-459e-84d3-069bd651755e", "query": "getting started with wandb", "matching_document": "## Quickstart\n\nInstall W&B and start tracking your machine learning experiments in minutes. \n\n## 1. Create an account and install W&B \n\nBefore you get started, make sure you create an account and install W&B: \n\n1. Sign up for a free account at and then log in to your wandb account.\n2. Install the wandb library on your machine in a Python 3 environment using `pip`. \n\nThe following code snippets demonstrate how to install and log into W&B using the W&B CLI and Python Library: \n\nInstall the CLI and Python library for interacting with the Weights and Biases API: \n\n```\npip install wandb\n\n``` \n\nInstall the CLI and Python library for interacting with the Weights and Biases API: \n\n```\n!pip install wandb\n\n``` \n\n## 2. Log in to W&B \n\nNext, log in to W&B: \n\n```\nwandb login\n\n``` \n\nOr if you are using W&B Server (including **Dedicated Cloud** or **Self-managed**): \n\n```\nwandb login --relogin --host=http://your-shared-local-host.com\n\n``` \n\n## 3. Start a run and track hyperparameters", "matching_document_document_id": "4c3d69bd-11c2-4856-87e5-bcc4fa96d3ab", "query_weight": 0.9290322580645161 }, { "query_id": "716fc38d-428d-4010-a3d6-f6f45c07f182", "query": "W&B artifact directory", "matching_document": "## Storage\n\nArtifact Storage \n\nW&B stores artifact files in a private Google Cloud Storage bucket located in the United States by default. All files are encrypted at rest and in transit. \n\nFor sensitive files, we recommend you set up Private Hosting or use reference artifacts. \n\nDuring training, W&B locally saves logs, artifacts, and configuration files in the following local directories: \n\n| File | Default location | To change default location set: |\n| --- | --- | --- |\n| logs | `./wandb` | `dir` in `wandb.init` or set the `WANDB_DIR` environment variable |\n| artifacts | `~/.cache/wandb` | the `WANDB_CACHE_DIR` environment variable |\n| configs | `~/.config/wandb` | the `WANDB_CONFIG_DIR` environment variable | \n\n:::caution\nDepending on the machine on `wandb` is initialized on, these default folders may not be located in a writeable part of the file system. This might trigger an error.\n::: \n\n### Clean up local artifact cache \n\n### How much storage does each artifact version use?", "matching_document_document_id": "2ec0a152-2b53-4183-8398-b61730217aac", "query_weight": 1.3935483870967742 }, { "query_id": "288d8b56-0d21-40cc-9562-0b06880b6846", "query": "What does setting 'resume' to 'allow' in 'wandb.init' do?", "matching_document": "## Resume Runs\n#### Resume Guidance\n##### Automatic and controlled resuming\n\nAutomatic resuming only works if the process is restarted on top of the same filesystem as the failed process. If you can't share a filesystem, we allow you to set the `WANDB_RUN_ID`: a globally unique string (per project) corresponding to a single run of your script. It must be no longer than 64 characters. All non-word characters will be converted to dashes. \n\n```\n# store this id to use it later when resuming\nid = wandb.util.generate\\_id()\nwandb.init(id=id, resume=\"allow\")\n# or via environment variables\nos.environ[\"WANDB\\_RESUME\"] = \"allow\"\nos.environ[\"WANDB\\_RUN\\_ID\"] = wandb.util.generate\\_id()\nwandb.init()\n\n``` \n\nIf you set `WANDB_RESUME` equal to `\"allow\"`, you can always set `WANDB_RUN_ID` to a unique string and restarts of the process will be handled automatically. If you set `WANDB_RESUME` equal to `\"must\"`, W&B will throw an error if the run to be resumed does not exist yet instead of auto-creating a new run.", "matching_document_document_id": "717baee4-2e53-4b5c-b345-41f09c0ff195", "query_weight": 1.0219354838709678 }, { "query_id": "02ddf8ce-24e5-45fb-a3f8-d96a994abfe7", "query": "Connecting Weights & Biases with LangChain", "matching_document": "## Prompts Quickstart\n### Use W&B Trace with LangChain\n#### 1. Set the LANGCHAIN\\_WANDB\\_TRACING environment variable\nFirst, set the LANGCHAIN\\_WANDB\\_TRACING environment variable to true. This will turn on automated Weights & Biases logging with LangChain: \n\n```\nimport os\n\n# turn on wandb logging for langchain\nos.environ[\"LANGCHAIN\\_WANDB\\_TRACING\"] = \"true\"\n\n``` \n\nThats it! Now any call to a LangChain LLM, Chain, Tool or Agent will be logged to Weights & Biases.", "matching_document_document_id": "6adad621-230b-42f0-badc-1738fef555a6", "query_weight": 1.2541935483870967 }, { "query_id": "8e750424-2ed5-4341-adee-726a1197846b", "query": "Getting started with WandB", "matching_document": "## Quickstart\n\nInstall W&B and start tracking your machine learning experiments in minutes. \n\n## 1. Create an account and install W&B \n\nBefore you get started, make sure you create an account and install W&B: \n\n1. Sign up for a free account at and then log in to your wandb account.\n2. Install the wandb library on your machine in a Python 3 environment using `pip`. \n\nThe following code snippets demonstrate how to install and log into W&B using the W&B CLI and Python Library: \n\nInstall the CLI and Python library for interacting with the Weights and Biases API: \n\n```\npip install wandb\n\n``` \n\nInstall the CLI and Python library for interacting with the Weights and Biases API: \n\n```\n!pip install wandb\n\n``` \n\n## 2. Log in to W&B \n\nNext, log in to W&B: \n\n```\nwandb login\n\n``` \n\nOr if you are using W&B Server (including **Dedicated Cloud** or **Self-managed**): \n\n```\nwandb login --relogin --host=http://your-shared-local-host.com\n\n``` \n\n## 3. Start a run and track hyperparameters", "matching_document_document_id": "4c3d69bd-11c2-4856-87e5-bcc4fa96d3ab", "query_weight": 0.9290322580645161 }, { "query_id": "52cb3ed4-1a3c-4318-8498-2df24897b994", "query": "Examples of wandb.save usage", "matching_document": "## Save & Restore Files\n### Saving Files\n#### Examples of `wandb.save`\n```\n# Save a model file from the current directory\nwandb.save(\"model.h5\")\n\n# Save all files that exist containing the substring \"ckpt\"\nwandb.save(\"../logs/\\*ckpt\\*\")\n\n# Save files starting with \"checkpoint\" as they're written to\nwandb.save(os.path.join(wandb.run.dir, \"checkpoint\\*\"))\n\n``` \n\n:::info\nW&B's local run directories are by default inside the `./wandb` directory relative to your script, and the path looks like `run-20171023_105053-3o4933r0` where `20171023_105053` is the timestamp and `3o4933r0` is the ID of the run. You can set the `WANDB_DIR` environment variable, or the `dir` keyword argument of `wandb.init`, to an absolute path and files will be written within that directory instead.\n:::", "matching_document_document_id": "0d3b29a7-20a4-489e-a16e-70827196337e", "query_weight": 0.7432258064516128 }, { "query_id": "c4480a5c-487d-4942-942d-794fbb911f6c", "query": "What features does Weights & Biases offer for LLM-powered application development?", "matching_document": "**Weights & Biases Prompts** is a suite of LLMOps tools built for the development of LLM-powered applications. \n\nUse W&B Prompts to visualize and inspect the execution flow of your LLMs, analyze the inputs and outputs of your LLMs, view the intermediate results and securely store and manage your prompts and LLM chain configurations. \n\n#### \ud83e\ude84 View Prompts In Action \n\n**In this notebook we will demostrate W&B Prompts:** \n\n* Using our 1-line LangChain integration\n* Using our Trace class when building your own LLM Pipelines \n\nSee here for the full W&B Prompts documentation \n\n## Installation \n\n```\n!pip install \"wandb>=0.15.4\" -qqq\n!pip install \"langchain>=0.0.218\" openai -qqq\n\n``` \n\n```\nimport langchain\nassert langchain.__version__ >= \"0.0.218\", \"Please ensure you are using LangChain v0.0.188 or higher\"\n\n``` \n\n## Setup \n\nThis demo requires that you have an OpenAI key \n\n# W&B Prompts \n\nW&B Prompts consists of three main components: \n\n**Trace table**: Overview of the inputs and outputs of a chain.", "matching_document_document_id": "5ccb89d6-6353-4b96-9920-060f32aad458", "query_weight": 1.207741935483871 }, { "query_id": "016f4cc8-09de-49c0-a4fa-4acc9c22a7de", "query": "How to properly delete an artifact in Weights & Biases?", "matching_document": "## Artifact\n### Methods\n#### `delete`\n\nView source \n\n```\ndelete(\ndelete\\_aliases: bool = (False)\n) -> None\n\n``` \n\nDelete an artifact and its files. \n\n| Arguments | |\n| --- | --- |\n| `delete_aliases` | If true, deletes all aliases associated with the artifact. Otherwise, this raises an exception if the artifact has existing aliases. | \n\n| Raises | |\n| --- | --- |\n| `ArtifactNotLoggedError` | if the artifact has not been logged | \n\n#### Examples: \n\nDelete all the \"model\" artifacts a run has logged: \n\n```\nruns = api.runs(path=\"my\\_entity/my\\_project\")\nfor run in runs:\nfor artifact in run.logged\\_artifacts():\nif artifact.type == \"model\":\nartifact.delete(delete\\_aliases=True)\n\n```", "matching_document_document_id": "542aac91-c6ad-4d2d-a1bd-fa6946db7698", "query_weight": 0.1 }, { "query_id": "bc1f94a7-7c60-49b8-bcd5-bf7177a7bd07", "query": "Weights & Biases sweep documentation", "matching_document": "\"\"\"Initialize a hyperparameter sweep.\n\n Search for hyperparameters that optimizes a cost function\n of a machine learning model by testing various combinations.\n\n Make note the unique identifier, `sweep_id`, that is returned.\n At a later step provide the `sweep_id` to a sweep agent.\n\n Args:\n sweep: The configuration of a hyperparameter search.\n (or configuration generator). See\n [Sweep configuration structure](https://docs.wandb.ai/guides/sweeps/define-sweep-configuration)\n for information on how to define your sweep.\n If you provide a callable, ensure that the callable does\n not take arguments and that it returns a dictionary that\n conforms to the W&B sweep config spec.\n entity: The username or team name where you want to send W&B\n runs created by the sweep to. Ensure that the entity you\n specify already exists. If you don't specify an entity,\n the run will be sent to your default entity,\n which is usually your username.\n project: The name of the project where W&B runs created from\n the sweep are sent to. If the project is not specified, the\n run is sent to a project labeled 'Uncategorized'.\n\n Returns:\n sweep_id: str. A unique identifier for the sweep.\n \"\"\"", "matching_document_document_id": "dde2c379-5417-4e62-9d30-034c0162fcc7", "query_weight": 0.1 }, { "query_id": "1e1f8ef8-a9e1-46b1-be63-3dc51ffc29e3", "query": "What data types and formats are supported for logging in Weights & Biases?", "matching_document": "## Data Types\n\nView source on GitHub \n\nThis module defines data types for logging rich, interactive visualizations to W&B. \n\nData types include common media types, like images, audio, and videos,\nflexible containers for information, like tables and HTML, and more. \n\nFor more on logging media, see our guide \n\nFor more on logging structured data for interactive dataset and model analysis,\nsee our guide to W&B Tables. \n\nAll of these special data types are subclasses of WBValue. All the data types\nserialize to JSON, since that is what wandb uses to save the objects locally\nand upload them to the W&B server. \n\n## Classes \n\n`class Audio`: Wandb class for audio clips. \n\n`class BoundingBoxes2D`: Format images with 2D bounding box overlays for logging to W&B. \n\n`class Graph`: Wandb class for graphs. \n\n`class Histogram`: wandb class for histograms. \n\n`class Html`: Wandb class for arbitrary html. \n\n`class Image`: Format images for logging to W&B.", "matching_document_document_id": "7cc5bc04-32c2-44e1-a102-18354f80ccb2", "query_weight": 1.8116129032258064 }, { "query_id": "b2f9049e-c5a8-4980-8739-04740771354f", "query": "Exporting Weights & Biases chart data via API", "matching_document": "## Import & Export Data\n### Export Data\nUse the Public API to export or update data that you have saved to W&B. Before using this API, you'll want to log data from your script \u2014 check the Quickstart for more details. \n\n**Use Cases for the Public API** \n\n* **Export Data**: Pull down a dataframe for custom analysis in a Jupyter Notebook. Once you have explored the data, you can sync your findings by creating a new analysis run and logging results, for example: `wandb.init(job_type=\"analysis\")`\n* **Update Existing Runs**: You can update the data logged in association with a W&B run. For example, you might want to update the config of a set of runs to include additional information, like the architecture or a hyperparameter that wasn't originally logged. \n\nSee the Generated Reference Docs for details on available functions. \n\n### Authentication \n\nAuthenticate your machine with your API key in one of two ways: \n\n1. Run `wandb login` on the command line and paste in your API key.\n2. Set the `WANDB_API_KEY` environment variable to your API key.", "matching_document_document_id": "6fab5684-679d-4e4c-baa9-1016ff28e256", "query_weight": 1.8116129032258064 }, { "query_id": "4209a9ee-dd2a-491a-bbda-8a3289a99c84", "query": "Customizing chart views in Weights & Biases", "matching_document": "## Custom Charts Walkthrough\n### 3. Customize the chart\nNow that looks pretty good, but I'd like to switch from a scatter plot to a line plot. Click **Edit** to change the Vega spec for this built in chart. Follow along in this workspace. \n\nI updated the Vega spec to customize the visualization: \n\n* add titles for the plot, legend, x-axis, and y-axis (set \u201ctitle\u201d for each field)\n* change the value of \u201cmark\u201d from \u201cpoint\u201d to \u201cline\u201d\n* remove the unused \u201csize\u201d field \n\nTo save this as a preset that you can use elsewhere in this project, click **Save as** at the top of the page. Here's what the result looks like, along with an ROC curve:", "matching_document_document_id": "2e1d68d2-2ae7-4aaa-af4a-9121f37d5d63", "query_weight": 1.8116129032258064 }, { "query_id": "cb54bed8-eff1-4b11-827d-1ff1cda9c23d", "query": "Steps to integrate LangChain with Weights & Biases", "matching_document": "## Prompts Quickstart\n### Use W&B Trace with LangChain\n#### 1. Set the LANGCHAIN\\_WANDB\\_TRACING environment variable\nFirst, set the LANGCHAIN\\_WANDB\\_TRACING environment variable to true. This will turn on automated Weights & Biases logging with LangChain: \n\n```\nimport os\n\n# turn on wandb logging for langchain\nos.environ[\"LANGCHAIN\\_WANDB\\_TRACING\"] = \"true\"\n\n``` \n\nThats it! Now any call to a LangChain LLM, Chain, Tool or Agent will be logged to Weights & Biases.", "matching_document_document_id": "6adad621-230b-42f0-badc-1738fef555a6", "query_weight": 1.2541935483870967 }, { "query_id": "bdeadbd4-fdde-47d1-abbe-7e2f0d13cb5a", "query": "wandb.plot.confusion_matrix visualization", "matching_document": "' \n\n# Confusion Matrix: Usage and Examples \n\nDescription: In this article, we review the usage and examples for a multi-class confusion matrix using Weights & Biases. \n\nBody: \n\n# Method: wandb.plot.confusion_matrix() \n\nLog a multi-class [confusion matrix](https://scikit-learn.org/stable/auto_examples/model_selection/plot_confusion_matrix.html) in one line: \n\n```\nwandb.log({\"conf_mat\" : wandb.plot.confusion_matrix(probs=None,\ny_true=ground_truth, preds=predictions,\nclass_names=class_names)})\n``` \n\nYou can log this wherever your code has access to: \n\n# Basic Usage \n\nIn this toy example, I finetune a CNN to predict one of 10 classes of living things in a photo (plants, animals, insects) while varying the number of epochs (E or pretrain_epochs) and the number of training examples (NT or num_train). I log a confusion matrix in each validation step, after each training epoch (so only the last confusion matrix at the end of training is visualized). \n\n# Powerful Interactions \n\n# Logging Details", "matching_document_document_id": "f3700051-8c7a-48b4-bf96-0e5fef0c6039", "query_weight": 0.8361290322580645 }, { "query_id": "5b159279-47e0-47b1-b528-c3965d367ad0", "query": "W&B sweep setup guide", "matching_document": "## Walkthrough\n\nSweeps Walkthrough \n\nThis page shows how to define, initialize, and run a sweep. There are four main steps: \n\n1. Set up your training code\n2. Define the search space with a sweep configuration\n3. Initialize the sweep\n4. Start the sweep agent \n\nCopy and paste the following code into a Jupyter Notebook or Python script: \n\n```\n# Import the W&B Python Library and log into W&B\nimport wandb\n\nwandb.login()\n\n# 1: Define objective/training function\ndef objective(config):\nscore = config.x\\*\\*3 + config.y\nreturn score\n\ndef main():\nwandb.init(project=\"my-first-sweep\")\nscore = objective(wandb.config)\nwandb.log({\"score\": score})\n\n# 2: Define the search space\nsweep\\_configuration = {\n\"method\": \"random\",\n\"metric\": {\"goal\": \"minimize\", \"name\": \"score\"},\n\"parameters\": {\n\"x\": {\"max\": 0.1, \"min\": 0.01},\n\"y\": {\"values\": [1, 3, 7]},\n},\n}\n\n# 3: Start the sweep\nsweep\\_id = wandb.sweep(sweep=sweep\\_configuration, project=\"my-first-sweep\")\n\nwandb.agent(sweep\\_id, function=main, count=10)\n\n```", "matching_document_document_id": "14e36cf6-fbf4-49be-b762-8b074da7dfa5", "query_weight": 0.1 }, { "query_id": "b24bfb9b-3027-454c-9f1d-642cdb42ea0d", "query": "collaboration in W&B", "matching_document": "## Teams\n\nUse W&B Teams as a central workspace for your ML team to build better models faster. \n\n* **Track all the experiments** your team has tried so you never duplicate work.\n* **Save and reproduce** previously trained models.\n* **Share progress** and results with your boss and collaborators.\n* **Catch regressions** and immediately get alerted when performance drops.\n* **Benchmark model performance** and compare model versions. \n\n## Create a collaborative team \n\n1. **Sign up or log in** to your free W&B account.\n2. Click **Invite Team** in the navigation bar.\n3. Create your team and invite collaborators. \n\n:::info\n**Note**: Only the admin of an organization can create a new team.\n::: \n\n## Create a Team Profile \n\nYou can customize your team's profile page to show an introduction and showcase reports and projects that are visible to the public or team members. Present reports, projects, and external links. \n\n## Remove Team members \n\n## Team Roles and Permissions \n\n## Team Trials", "matching_document_document_id": "878a2967-599f-4092-9854-920a07a186ba", "query_weight": 1.3470967741935482 }, { "query_id": "25d06fe6-b677-4781-a899-599000f3473c", "query": "Methods to download W&B artifacts without API key", "matching_document": "## Download and use artifacts\n#### Download and use an artifact stored on W&B\nUse the object returned to download all the contents of the artifact: \n\n```\ndatadir = artifact.download()\n\n``` \n\nYou can optionally pass a path to the root parameter to download the contents of the artifact to a specific directory. For more information, see the Python SDK Reference Guide. \n\nUse the `get_path` method to download only subset of files: \n\n```\npath = artifact.get\\_path(name)\n\n``` \n\nThis fetches only the file at the path `name`. It returns an `Entry` object with the following methods: \n\n* `Entry.download`: Downloads file from the artifact at path `name`\n* `Entry.ref`: If the entry was stored as a reference using `add_reference`, returns the URI \n\nReferences that have schemes that W&B knows how to handle can be downloaded just like artifact files. For more information, see Track external files. \n\nFirst, import the W&B SDK. Next, create an artifact from the Public API Class. Provide the entity, project, artifact, and alias associated with that artifact:", "matching_document_document_id": "a42e4236-eca7-4763-a693-b44e8187ca6f", "query_weight": 1.2541935483870967 }, { "query_id": "536b4db3-7a33-44be-8066-6f4bab9ca8d4", "query": "Weights & Biases capabilities", "matching_document": "## Weights & Biases overview\nW&B is a platform that helps data scientists track their models, datasets, system information and more. With a few lines of code, you can start tracking everything about these features. It's free for personal use. Team use is normally a paid utility, but teams for academic purposes are free. You can use W&B with your favourite framework, like TensorFlow, Keras, PyTorch, Sklearn, fastai and many others. \n\nAll tracking information is sent to a dedicated project page on the W&B UI, where you can open high quality visualizations, aggregate information and compare models or parameters. One of the advantages of remotely storing the experiment\u2019s information is that it is easy to collaborate on the same project and share the results with your teammates. \n\nW&B provides 4 useful tools: \n\n* Dashboard: Experiment tracking\n* Artifacts: Dataset versioning, model versioning\n* Sweeps: Hyperparameter optimization\n* Reports: Save and share reproducible findings", "matching_document_document_id": "8c1600ad-9b6e-45cd-86ae-03afd8f80332", "query_weight": 0.9754838709677419 }, { "query_id": "4ecb5aed-7938-4cdc-979c-6c4758d93cea", "query": "How to correctly use markdown in Weights & Biases reports?", "matching_document": "## Edit a report\n#### Markdown\nAdd markdown to your report interactively with the App UI or with the W&B SDK. \n\nEnter a forward slash (`/`) in the report to display a dropdown menu. From the dropdown choose **Markdown**. \n\nUse the `wandb.apis.reports.MarkdownBlock` Class to create a markdown block programmatically. Pass a string to the `text` parameter: \n\n```\nimport wandb\nimport wandb.apis.reports as wr\n\nreport = wr.Report(project=\"report-editing\")\n\nreport.blocks = [\nwr.MarkdownBlock(text=\"Markdown cell with \\*italics\\* and \\*\\*bold\\*\\* and $e=mc^2$\")\n]\n\n``` \n\nThis will render a markdown block similar to:", "matching_document_document_id": "c651c051-93c3-41b2-bfc8-844d8e467a7f", "query_weight": 0.1 }, { "query_id": "0c4851cb-6c50-4458-8238-e78583d0ec8b", "query": "troubleshooting wandb network issues", "matching_document": "## Troubleshooting\n#### How do I deal with network issues?\n\nIf you're seeing SSL or network errors:`wandb: Network error (ConnectionError), entering retry loop`. You can try a couple of different approaches to solving this issue: \n\n1. Upgrade your SSL certificate. If you're running the script on an Ubuntu server, run `update-ca-certificates` We can't sync training logs without a valid SSL certificate because it's a security vulnerability.\n2. If your network is flaky, run training in offline mode and sync the files to us from a machine that has Internet access.\n3. Try running W&B Private Hosting, which operates on your machine and doesn't sync files to our cloud servers. \n\n`SSL CERTIFICATE_VERIFY_FAILED`: this error could be due to your company's firewall. You can set up local CAs and then use: \n\n`export REQUESTS_CA_BUNDLE=/etc/ssl/certs/ca-certificates.crt`", "matching_document_document_id": "eba9b6dc-4582-4539-bf24-39b10a2e6960", "query_weight": 1.7651612903225806 }, { "query_id": "07d696f0-c0df-47e3-a780-1f045d51798a", "query": "How can W&B reports be shared with users without a W&B account?", "matching_document": "## Collaborate on reports\n\nCollaborate and Share W&B Reports \n\nOnce you have saved a report, you can select the **Share** button to collaborate. A draft copy of the report is created when you select the **Edit** button. Draft reports auto-save. Select **Save to report** to publish your changes to the shared report. \n\nA warning notification will appear if an edit conflict occurs. This can occur if you and another collaborator edit the same report at the same time. The warning notification will guide you to resolve potential edit conflicts. \n\n### Comment on reports \n\nClick the comment button on a panel in a report to add a comment directly to that panel. \n\n### Who can edit and share reports? \n\nReports that are created within an individual's private project is only visible to that user. The user can share their project to a team or to the public.", "matching_document_document_id": "5bab537b-05eb-4ac3-9db7-27a98adcbed0", "query_weight": 0.6038709677419355 }, { "query_id": "54c7eeb2-93e4-4d41-bb07-7d949cc654bb", "query": "Weights & Biases API", "matching_document": "## Api\n\nView source on GitHub \n\nUsed for querying the wandb server. \n\n```\nApi(\noverrides=None,\ntimeout: Optional[int] = None,\napi\\_key: Optional[str] = None\n) -> None\n\n``` \n\n#### Examples: \n\nMost common way to initialize \n\n```\n>>> wandb.Api()\n\n``` \n\n| Arguments | |\n| --- | --- |\n| `overrides` | (dict) You can set `base_url` if you are using a wandb server other than . You can also set defaults for `entity`, `project`, and `run`. | \n\n| Attributes | |\n| --- | --- |\n| | | \n\n## Methods \n\n### `artifact` \n\nView source \n\n```\nartifact(\nname, type=None\n)\n\n``` \n\nReturn a single artifact by parsing path in the form `entity/project/name`. \n\n| Arguments | |\n| --- | --- |\n| `name` | (str) An artifact name. May be prefixed with entity/project. Valid names can be in the following forms: name:version name:alias |\n| `type` | (str, optional) The type of artifact to fetch. | \n\n| Returns | |\n| --- | --- |\n| A `Artifact` object. | | \n\n### `artifact_type` \n\nView source \n\n### `flush`", "matching_document_document_id": "8d4ccf38-8646-42b5-abfa-b46e0c4ca9f5", "query_weight": 0.1 }, { "query_id": "359e38cf-010f-4343-991c-137091106604", "query": "How to correctly log a confusion matrix chart in Weights & Biases?", "matching_document": "' \n\n# Confusion Matrix: Usage and Examples \n\nDescription: In this article, we review the usage and examples for a multi-class confusion matrix using Weights & Biases. \n\nBody: \n\n# Method: wandb.plot.confusion_matrix() \n\nLog a multi-class [confusion matrix](https://scikit-learn.org/stable/auto_examples/model_selection/plot_confusion_matrix.html) in one line: \n\n```\nwandb.log({\"conf_mat\" : wandb.plot.confusion_matrix(probs=None,\ny_true=ground_truth, preds=predictions,\nclass_names=class_names)})\n``` \n\nYou can log this wherever your code has access to: \n\n# Basic Usage \n\nIn this toy example, I finetune a CNN to predict one of 10 classes of living things in a photo (plants, animals, insects) while varying the number of epochs (E or pretrain_epochs) and the number of training examples (NT or num_train). I log a confusion matrix in each validation step, after each training epoch (so only the last confusion matrix at the end of training is visualized). \n\n# Powerful Interactions \n\n# Logging Details", "matching_document_document_id": "f3700051-8c7a-48b4-bf96-0e5fef0c6039", "query_weight": 0.1 }, { "query_id": "9437cc2b-81b0-4796-866d-9cee78273dfb", "query": "How to download W&B artifacts without an API key?", "matching_document": "## Download and use artifacts\n#### Download and use an artifact stored on W&B\nUse the object returned to download all the contents of the artifact: \n\n```\ndatadir = artifact.download()\n\n``` \n\nYou can optionally pass a path to the root parameter to download the contents of the artifact to a specific directory. For more information, see the Python SDK Reference Guide. \n\nUse the `get_path` method to download only subset of files: \n\n```\npath = artifact.get\\_path(name)\n\n``` \n\nThis fetches only the file at the path `name`. It returns an `Entry` object with the following methods: \n\n* `Entry.download`: Downloads file from the artifact at path `name`\n* `Entry.ref`: If the entry was stored as a reference using `add_reference`, returns the URI \n\nReferences that have schemes that W&B knows how to handle can be downloaded just like artifact files. For more information, see Track external files. \n\nFirst, import the W&B SDK. Next, create an artifact from the Public API Class. Provide the entity, project, artifact, and alias associated with that artifact:", "matching_document_document_id": "a42e4236-eca7-4763-a693-b44e8187ca6f", "query_weight": 1.2541935483870967 }, { "query_id": "00edcf86-8cd1-4494-9731-05afb341f1bb", "query": "Example of using Weights & Biases with PyTorch", "matching_document": "## \ud83d\udd25 = W&B \u2795 PyTorch\n\nUse Weights & Biases for machine learning experiment tracking, dataset versioning, and project collaboration. \n\n## What this notebook covers: \n\nWe show you how to integrate Weights & Biases with your PyTorch code to add experiment tracking to your pipeline. \n\n## The resulting interactive W&B dashboard will look like: \n\n## In pseudocode, what we'll do is: \n\n```\n# import the library\nimport wandb\n\n# start a new experiment\nwandb.init(project=\"new-sota-model\")\n\n#\u2003capture a dictionary of hyperparameters with config\nwandb.config = {\"learning\\_rate\": 0.001, \"epochs\": 100, \"batch\\_size\": 128}\n\n# set up model and data\nmodel, dataloader = get\\_model(), get\\_data()\n\n# optional: track gradients\nwandb.watch(model)\n\nfor batch in dataloader:\nmetrics = model.training\\_step()\n#\u2003log metrics inside your training loop to visualize model performance\nwandb.log(metrics)\n\n# optional: save model at the end\nmodel.to\\_onnx()\nwandb.save(\"model.onnx\")\n\n``` \n\n## Follow along with a video tutorial!", "matching_document_document_id": "ee62983f-f238-4b4a-bde8-6b4c068d7918", "query_weight": 1.2541935483870967 }, { "query_id": "dd200979-e882-4c7a-abe7-ebc4925aa9d1", "query": "W&B permissions", "matching_document": "## Teams\n### Team Roles and Permissions\n* **Admin**: Team admins can add and remove other admins or team members. They have permissions to modify all projects and full deletion permissions. This includes, but is not limited to, deleting runs, projects, artifacts, and sweeps.\n* **Member**: A regular member of the team. A team member is invited by email by the team admin. A team member cannot invite other members. Team members can only delete runs and sweep runs created by that member. Suppose you have two members A and B. Member B moves a Run from team B's project to a different project owned by Member A. Member A can not delete the Run Member B moved to Member A's project. Only the member that creates the Run, or the team admin, can delete the run.\n* **Service (Enterprise-only feature)**: A service worker, an API key useful for using W&B with your run automation tools. If you use the API key from a service account for your team, make sure to set the environment variable **WANDB\\_USERNAME** to attribute runs to the correct user. See more on the relevant behavior below.\n* **View-Only (Enterprise-only feature)**: View-Only members can view assets within the team such as runs, reports, and workspaces. They can follow and comment on reports, but they can not create, edit, or delete project overview, reports, or runs. View-Only members do not have an API key.\n* **Custom roles (Enterprise-only feature)**: Custom roles allow organization admins to compose new roles by inheriting from the above **View-Only** or **Member** roles, and adding additional permissions to achieve fine-grained access control. Team admins can then assign any of those custom roles to users in their respective teams. Refer to this article for details.", "matching_document_document_id": "d04c7473-013b-4afa-8c71-ecb44bc24666", "query_weight": 1.3470967741935482 }, { "query_id": "202a2fae-4f72-4375-926c-77959c0f5025", "query": "What are the key features of WandB?", "matching_document": "## Weights & Biases overview\nW&B is a platform that helps data scientists track their models, datasets, system information and more. With a few lines of code, you can start tracking everything about these features. It's free for personal use. Team use is normally a paid utility, but teams for academic purposes are free. You can use W&B with your favourite framework, like TensorFlow, Keras, PyTorch, Sklearn, fastai and many others. \n\nAll tracking information is sent to a dedicated project page on the W&B UI, where you can open high quality visualizations, aggregate information and compare models or parameters. One of the advantages of remotely storing the experiment\u2019s information is that it is easy to collaborate on the same project and share the results with your teammates. \n\nW&B provides 4 useful tools: \n\n* Dashboard: Experiment tracking\n* Artifacts: Dataset versioning, model versioning\n* Sweeps: Hyperparameter optimization\n* Reports: Save and share reproducible findings", "matching_document_document_id": "8c1600ad-9b6e-45cd-86ae-03afd8f80332", "query_weight": 0.9290322580645161 }, { "query_id": "826bfc5f-fd23-4e9d-afcb-39d5b14fc54b", "query": "Integrating AWS IAM with SageMaker for secure training jobs", "matching_document": "## Set up for SageMaker\n### Prerequisites\n1. **Setup SageMaker in your AWS account.** See the SageMaker Developer guide for more information.\n2. **Create an Amazon ECR repository** to store images you want to execute on Amazon SageMaker. See the Amazon ECR documentation for more information.\n3. **Create an Amazon S3 buckets** to store SageMaker inputs and outputs for your SageMaker training jobs. See the Amazon S3 documentation for more information. Make note of the S3 bucket URI and directory.\n4. **Create IAM execution role.** The role used in the SageMaker training job requires the following permissions to work. These permissions allow for logging events, pulling from ECR, and interacting with input and output buckets. (Note: if you already have this role for SageMaker training jobs, you do not need to create it again.)\nIAM role policy\n```\n{\n\"Version\": \"2012-10-17\",\n\"Statement\": [\n{\n\"Effect\": \"Allow\",\n\"Action\": [\n\"cloudwatch:PutMetricData\",\n\"logs:CreateLogStream\",\n\"logs:PutLogEvents\",\n\"logs:CreateLogGroup\",\n\"logs:DescribeLogStreams\",\n\"ecr:GetAuthorizationToken\"\n],\n\"Resource\": \"\\*\"\n},\n{\n\"Effect\": \"Allow\",\n\"Action\": [\n\"s3:ListBucket\"\n],\n\"Resource\": [\n\"arn:aws:s3:::\"\n]\n},\n{\n\"Effect\": \"Allow\",\n\"Action\": [\n\"s3:GetObject\",\n\"s3:PutObject\"\n],\n\"Resource\": [\n\"arn:aws:s3:::/\",\n\"arn:aws:s3:::/\"\n]\n},\n{\n\"Effect\": \"Allow\",\n\"Action\": [\n\"ecr:BatchCheckLayerAvailability\",\n\"ecr:GetDownloadUrlForLayer\",\n\"ecr:BatchGetImage\"\n],\n\"Resource\": \"arn:aws:ecr:::repository/\"\n}\n]\n}\n\n```", "matching_document_document_id": "455be611-540d-4ed2-a34d-bd233f9e98d6", "query_weight": 1.0683870967741935 }, { "query_id": "ee97e5d3-a8f1-4a93-8051-9c1723393cd9", "query": "managing team permissions on Weights & Biases", "matching_document": "## Teams\n### Team Roles and Permissions\n* **Admin**: Team admins can add and remove other admins or team members. They have permissions to modify all projects and full deletion permissions. This includes, but is not limited to, deleting runs, projects, artifacts, and sweeps.\n* **Member**: A regular member of the team. A team member is invited by email by the team admin. A team member cannot invite other members. Team members can only delete runs and sweep runs created by that member. Suppose you have two members A and B. Member B moves a Run from team B's project to a different project owned by Member A. Member A can not delete the Run Member B moved to Member A's project. Only the member that creates the Run, or the team admin, can delete the run.\n* **Service (Enterprise-only feature)**: A service worker, an API key useful for using W&B with your run automation tools. If you use the API key from a service account for your team, make sure to set the environment variable **WANDB\\_USERNAME** to attribute runs to the correct user. See more on the relevant behavior below.\n* **View-Only (Enterprise-only feature)**: View-Only members can view assets within the team such as runs, reports, and workspaces. They can follow and comment on reports, but they can not create, edit, or delete project overview, reports, or runs. View-Only members do not have an API key.\n* **Custom roles (Enterprise-only feature)**: Custom roles allow organization admins to compose new roles by inheriting from the above **View-Only** or **Member** roles, and adding additional permissions to achieve fine-grained access control. Team admins can then assign any of those custom roles to users in their respective teams. Refer to this article for details.", "matching_document_document_id": "d04c7473-013b-4afa-8c71-ecb44bc24666", "query_weight": 2.136774193548387 }, { "query_id": "a993d39c-a789-4530-988e-3b0be317c152", "query": "Ensuring correct data type for W&B logging", "matching_document": "## Data Types\n\nView source on GitHub \n\nThis module defines data types for logging rich, interactive visualizations to W&B. \n\nData types include common media types, like images, audio, and videos,\nflexible containers for information, like tables and HTML, and more. \n\nFor more on logging media, see our guide \n\nFor more on logging structured data for interactive dataset and model analysis,\nsee our guide to W&B Tables. \n\nAll of these special data types are subclasses of WBValue. All the data types\nserialize to JSON, since that is what wandb uses to save the objects locally\nand upload them to the W&B server. \n\n## Classes \n\n`class Audio`: Wandb class for audio clips. \n\n`class BoundingBoxes2D`: Format images with 2D bounding box overlays for logging to W&B. \n\n`class Graph`: Wandb class for graphs. \n\n`class Histogram`: wandb class for histograms. \n\n`class Html`: Wandb class for arbitrary html. \n\n`class Image`: Format images for logging to W&B.", "matching_document_document_id": "7cc5bc04-32c2-44e1-a102-18354f80ccb2", "query_weight": 1.2541935483870967 }, { "query_id": "eb44adae-8400-4e8a-9b1b-ced94397e773", "query": "wandb Table", "matching_document": "## Table\n\nView source on GitHub \n\nThe Table class used to display and analyze tabular data. \n\n```\nTable(\ncolumns=None, data=None, rows=None, dataframe=None, dtype=None, optional=(True),\nallow\\_mixed\\_types=(False)\n)\n\n``` \n\nUnlike traditional spreadsheets, Tables support numerous types of data:\nscalar values, strings, numpy arrays, and most subclasses of `wandb.data_types.Media`.\nThis means you can embed `Images`, `Video`, `Audio`, and other sorts of rich, annotated media\ndirectly in Tables, alongside other traditional scalar values. \n\nThis class is the primary class used to generate the Table Visualizer\nin the UI: . \n\nTables can be constructed with initial data using the `data` or\n`dataframe` parameters:", "matching_document_document_id": "ef293fdf-142d-4a6e-b668-514932593e8f", "query_weight": 0.1 }, { "query_id": "d5103a7a-0aa8-4242-98bd-69ab719976a9", "query": "How to integrate Weights & Biases with NLP models?", "matching_document": "## Optimize \ud83e\udd17 Hugging Face models with Weights & Biases\n\nHugging Face provides tools to quickly train neural networks for NLP (Natural Language Processing) on any task (classification, translation, question answering, etc) and any dataset with PyTorch and TensorFlow 2.0. \n\nCoupled with Weights & Biases integration, you can quickly train and monitor models for full traceability and reproducibility without any extra line of code! You just need to install the library, sign in, and your experiments will automatically be logged: \n\n```\npip install wandb\nwandb login\n\n``` \n\n**Note**: To enable logging to W&B, set `report_to` to `wandb` in your `TrainingArguments` or script. \n\nW&B integration with \ud83e\udd17 Hugging Face can automatically:\n\\* log your configuration parameters\n\\* log your losses and metrics\n\\* log gradients and parameter distributions\n\\* log your model\n\\* keep track of your code\n\\* log your system metrics (GPU, CPU, memory, temperature, etc) \n\n## Here's what the W&B interactive dashboard will look like:", "matching_document_document_id": "b42d7afe-6db5-4ce4-a388-988c7feb33e8", "query_weight": 0.9754838709677419 }, { "query_id": "e1e66cef-cef5-485f-b3ab-a6c3fb46f844", "query": "How to effectively use Weights & Biases for dataset versioning", "matching_document": "## Dataset Versioning\n### Core Artifacts features\n1. **Upload**: Start tracking and versioning any data (files or directories) with `run.log_artifact()`. You can also track datasets in a remote filesystem (e.g. cloud storage in S3 or GCP) by reference, using a link or URI instead of the raw contents.\n2. **Version**: Define an artifact by giving it a type (`\"raw_data\"`, `\"preprocessed_data\"`, `\"balanced_data\"`) and a name (`\"imagenet_cats_10K\"`). When you log the same name again, W&B automatically creates a new version of the artifact with the latest contents.\n3. **Alias**: Set an alias like `\"best\"` or `\"production\"` to highlight the important versions in a lineage of artifacts.\n4. **Compare**: Select any two versions to browse the contents side-by-side. We're also working on a tool for dataset visualization, learn more here.\n5. **Download**: Obtain a local copy of the artifact or verify the contents by reference. \n\nFor more detail on these features, check out Artifacts how it works.", "matching_document_document_id": "7407e418-a665-438f-b605-d5dc12e4240e", "query_weight": 1.3006451612903225 }, { "query_id": "6e72152a-9085-45c9-87da-8ca6016204a6", "query": "Linking runs to model registry in Weights & Biases", "matching_document": "## Model registry\n### How it works\n1. **Log a model version**: In your training script, add a few lines of code to save the model files as an artifact to W&B.\n2. **Compare performance**: Check live charts to compare the metrics and sample predictions from model training and validation. Identify which model version performed the best.\n3. **Link to registry**: Bookmark the best model version by linking it to a registered model, either programmatically in Python or interactively in the W&B UI. \n\nThe following code snippet demonstrates how to log and link a model to the Model Registry: \n\n```python showLineNumbers\nimport wandb\nimport random\n\n# Start a new W&B run\n\nrun = wandb.init(project=\"models\\_quickstart\")\n\n# Simulate logging model metrics\n\nrun.log({\"acc\": random.random()})\n\n# Create a simulated model file\n\nwith open(\"my\\_model.h5\", \"w\") as f:\nf.write(\"Model: \" + str(random.random()))\n\n# Log and link the model to the Model Registry\n\nrun.link\\_model(path=\"./my\\_model.h5\", registered\\_model\\_name=\"MNIST\")\n\nrun.finish()\n```", "matching_document_document_id": "c8056eb1-14f6-47e4-82a9-b839839eb916", "query_weight": 1.0219354838709678 }, { "query_id": "52522546-b944-4fc6-bdb5-6d420544c7a8", "query": "wandb.init resume parameter", "matching_document": "| `resume` | (bool, str, optional) Sets the resuming behavior. Options: `\"allow\"`, `\"must\"`, `\"never\"`, `\"auto\"` or `None`. Defaults to `None`. Cases: - `None` (default): If the new run has the same ID as a previous run, this run overwrites that data. - `\"auto\"` (or `True`): if the previous run on this machine crashed, automatically resume it. Otherwise, start a new run. - `\"allow\"`: if id is set with `init(id=\"UNIQUE_ID\")` or `WANDB_RUN_ID=\"UNIQUE_ID\"` and it is identical to a previous run, wandb will automatically resume the run with that id. Otherwise, wandb will start a new run. - `\"never\"`: if id is set with `init(id=\"UNIQUE_ID\")` or `WANDB_RUN_ID=\"UNIQUE_ID\"` and it is identical to a previous run, wandb will crash. - `\"must\"`: if id is set with `init(id=\"UNIQUE_ID\")` or `WANDB_RUN_ID=\"UNIQUE_ID\"` and it is identical to a previous run, wandb will automatically resume the run with the id. Otherwise, wandb will crash. See our guide to resuming runs for more. |\n| `reinit` | (bool, optional) Allow multiple `wandb.init()` calls in the same process. (default: `False`) |\n| `magic` | (bool, dict, or str, optional) The bool controls whether we try to auto-instrument your script, capturing basic details of your run without you having to add more wandb code. (default: `False`) You can also pass a dict, json string, or yaml filename. |\n| `config_exclude_keys` | (list, optional) string keys to exclude from `wandb.config`. |\n| `config_include_keys` | (list, optional) string keys to include in `wandb.config`. |", "matching_document_document_id": "01c9bdf0-fd26-4350-8de1-e75588077941", "query_weight": 1.0219354838709678 }, { "query_id": "7e77cee7-47f6-4059-a155-69adb2e75994", "query": "wandb Table pandas DataFrame integration", "matching_document": "## Plotting with Pandas and Weights & Biases: Step-by-Step Guide\n### Integration of Weights & Biases With Pandas for Interactive and Collaborative Data Plotting\n\nIt is also extremely simple to log any Pandas dataframe to a workspace by converting it into a W&B Table: \n\n```\nimport wandb\nimport pandas as pd\n\n# Read our CSV into a new DataFrame\npandas_dataframe = pd.read_csv(\"data.csv\")\n\n# Convert the DataFrame into a W&B Table\nwandb_table = wandb.Table(dataframe=pandas_dataframe)\n\n# Add the table to an Artifact to increase the row\nwandb_table_artifact = wandb.Artifact(\n\"wandb_artifact\",\ntype=\"dataset\")\nwandb_table_artifact.add(wandb_table, \"table\")\n\n# Log the raw csv file within an artifact to preserve our data\nwandb_table_artifact.add_file(\"data.csv\")\n\n# Start a W&B run to log data\nrun = wandb.init(project=\"...\")\n\n# Log the table to visualize with a run...\nrun.log({\"data\": wandb_table})\n\n# and Log as an Artifact\nrun.log_artifact(wandb_table_artifact)\n```", "matching_document_document_id": "b584a5a0-834b-4d77-b8e3-86ce64c829d1", "query_weight": 0.8361290322580645 }, { "query_id": "4cb63fe8-4ae5-45d5-8076-456af9579401", "query": "log images with Wandb", "matching_document": "## PyTorch Lightning\n### Using PyTorch Lightning's `WandbLogger`\n#### Log images, text and more\n\nThe `WandbLogger` has `log_image`, `log_text` and `log_table` methods for logging media. \n\nYou can also directly call `wandb.log` or `trainer.logger.experiment.log` to log other media types such as Audio, Molecules, Point Clouds, 3D Objects and more. \n\n```\n# using tensors, numpy arrays or PIL images\nwandb\\_logger.log\\_image(key=\"samples\", images=[img1, img2])\n\n# adding captions\nwandb\\_logger.log\\_image(key=\"samples\", images=[img1, img2], caption=[\"tree\", \"person\"])\n\n# using file path\nwandb\\_logger.log\\_image(key=\"samples\", images=[\"img\\_1.jpg\", \"img\\_2.jpg\"])\n\n# using .log in the trainer\ntrainer.logger.experiment.log({\n\"samples\": [wandb.Image(img, caption=caption)\nfor (img, caption) in my\\_images]\n})\n\n```", "matching_document_document_id": "8ea07346-5ed1-4f88-8173-091e69c9fea8", "query_weight": 0.9754838709677419 }, { "query_id": "7e8b06b3-9a1d-4d3a-9fa3-36cb78c25ce4", "query": "W&B sweeps hyperparameter constraints", "matching_document": "## Sweep configuration options\n### `parameters`\n| Search constraint | Description |\n| --- | --- |\n| `values` | Specifies all valid values for this hyperparameter. Compatible with `grid`. |\n| `value` | Specifies the single valid value for this hyperparameter. Compatible with `grid`. |\n| `distribution` | Specify a probability distribution. See the note following this table for information on default values. |\n| `probabilities` | Specify the probability of selecting each element of `values` when using `random`. |\n| `min`, `max` | (`int`or `float`) Maximum and minimum values. If `int`, for `int_uniform` -distributed hyperparameters. If `float`, for `uniform` -distributed hyperparameters. |\n| `mu` | (`float`) Mean parameter for `normal` - or `lognormal` -distributed hyperparameters. |\n| `sigma` | (`float`) Standard deviation parameter for `normal` - or `lognormal` -distributed hyperparameters. |\n| `q` | (`float`) Quantization step size for quantized hyperparameters. |\n| `parameters` | Nest other parameters inside a root level parameter. |", "matching_document_document_id": "c4741ef2-3ba1-4f32-945f-0450e2171423", "query_weight": 0.8825806451612903 }, { "query_id": "9b289bbf-545e-4d12-a8f2-52da244b2763", "query": "Weights & Biases LLM integration", "matching_document": "**Weights & Biases Prompts** is a suite of LLMOps tools built for the development of LLM-powered applications. \n\nUse W&B Prompts to visualize and inspect the execution flow of your LLMs, analyze the inputs and outputs of your LLMs, view the intermediate results and securely store and manage your prompts and LLM chain configurations. \n\n#### \ud83e\ude84 View Prompts In Action \n\n**In this notebook we will demostrate W&B Prompts:** \n\n* Using our 1-line LangChain integration\n* Using our Trace class when building your own LLM Pipelines \n\nSee here for the full W&B Prompts documentation \n\n## Installation \n\n```\n!pip install \"wandb>=0.15.4\" -qqq\n!pip install \"langchain>=0.0.218\" openai -qqq\n\n``` \n\n```\nimport langchain\nassert langchain.__version__ >= \"0.0.218\", \"Please ensure you are using LangChain v0.0.188 or higher\"\n\n``` \n\n## Setup \n\nThis demo requires that you have an OpenAI key \n\n# W&B Prompts \n\nW&B Prompts consists of three main components: \n\n**Trace table**: Overview of the inputs and outputs of a chain.", "matching_document_document_id": "5ccb89d6-6353-4b96-9920-060f32aad458", "query_weight": 1.207741935483871 }, { "query_id": "4e3c2935-e626-4c0c-8594-e1fcbacd602c", "query": "Training LLMs with Weights & Biases", "matching_document": "--- \n\n## displayed\\_sidebar: default \n\n# Hugging Face Autotrain \n\n\ud83e\udd17 AutoTrain is a no-code tool for training state-of-the-art models for Natural Language Processing (NLP) tasks, for Computer Vision (CV) tasks, and for Speech tasks and even for Tabular tasks. \n\nWeights & Biases is directly integrated into \ud83e\udd17 AutoTrain, providing experiment tracking and config management. It's as easy as using a single parameter in the CLI command for your experiments! \n\n| |\n| --- |\n| **An example of how the metrics of your experiment are logged.** | \n\n## Getting Started \n\nFirst, we need to install `autotrain-advanced` and `wandb`. \n\n```\npip install --upgrade autotrain-advanced wandb\n\n``` \n\n```\n!pip install --upgrade autotrain-advanced wandb\n\n``` \n\n## Getting Started: Fine-tuning an LLM \n\nTo demonstrate these changes we will fine-tune an LLM on a math dataset and try achieving SoTA result in `pass@1` on the GSM8k Benchmarks. \n\n### Preparing the Dataset \n\n### Training using Autotrain Advanced \n\n## More Resources", "matching_document_document_id": "bbbde78c-933b-41a1-b8a8-8e995f5b6523", "query_weight": 1.207741935483871 }, { "query_id": "05e470f7-cb15-48ad-ae85-075b6e4ec97d", "query": "Weights & Biases LangChain integration", "matching_document": "## Prompts Quickstart\n### Use W&B Trace with LangChain\n#### 1. Set the LANGCHAIN\\_WANDB\\_TRACING environment variable\nFirst, set the LANGCHAIN\\_WANDB\\_TRACING environment variable to true. This will turn on automated Weights & Biases logging with LangChain: \n\n```\nimport os\n\n# turn on wandb logging for langchain\nos.environ[\"LANGCHAIN\\_WANDB\\_TRACING\"] = \"true\"\n\n``` \n\nThats it! Now any call to a LangChain LLM, Chain, Tool or Agent will be logged to Weights & Biases.", "matching_document_document_id": "6adad621-230b-42f0-badc-1738fef555a6", "query_weight": 1.2541935483870967 }, { "query_id": "754784ab-8a9f-46a1-96a2-cc11183045f9", "query": "Step by step guide to start with wandb", "matching_document": "## Quickstart\n\nInstall W&B and start tracking your machine learning experiments in minutes. \n\n## 1. Create an account and install W&B \n\nBefore you get started, make sure you create an account and install W&B: \n\n1. Sign up for a free account at and then log in to your wandb account.\n2. Install the wandb library on your machine in a Python 3 environment using `pip`. \n\nThe following code snippets demonstrate how to install and log into W&B using the W&B CLI and Python Library: \n\nInstall the CLI and Python library for interacting with the Weights and Biases API: \n\n```\npip install wandb\n\n``` \n\nInstall the CLI and Python library for interacting with the Weights and Biases API: \n\n```\n!pip install wandb\n\n``` \n\n## 2. Log in to W&B \n\nNext, log in to W&B: \n\n```\nwandb login\n\n``` \n\nOr if you are using W&B Server (including **Dedicated Cloud** or **Self-managed**): \n\n```\nwandb login --relogin --host=http://your-shared-local-host.com\n\n``` \n\n## 3. Start a run and track hyperparameters", "matching_document_document_id": "4c3d69bd-11c2-4856-87e5-bcc4fa96d3ab", "query_weight": 0.9290322580645161 }, { "query_id": "5c32ebc8-38c7-4215-a68e-604c2987162a", "query": "How can Weights & Biases integrate with large language models?", "matching_document": "## OpenAI and Weights & Biases\n\nWeights & Biases has 2 OpenAI integrations to help you train better models faster and keep track of your prompts and generations: \n\n## OpenAI ChatGPT-3.5 and GPT-4 fine-tuning \n\nWith Weights & Biases you can log your OpenAI model's fine-tuning metrics and configuration to Weights & Biases to analyse and understand the performance of your newly fine-tuned models and share the results with your colleagues. \n\nSee the **W&B and OpenAI Fine-Tuning docs here** to learn more \n\n## OpenAI Python SDK API \n\nLog requests, responses, token counts and model metadata with 1 line of code for all OpenAI models \n\nSee the **W&B and OpenAI API section docs** to learn more", "matching_document_document_id": "6036811a-54cc-4eea-9c18-5a2b8190b5d4", "query_weight": 0.9754838709677419 }, { "query_id": "7b04a3db-ec4a-412a-8d73-4771064c5def", "query": "dataset versioning Weights & Biases", "matching_document": "## Dataset Versioning\n### Core Artifacts features\n1. **Upload**: Start tracking and versioning any data (files or directories) with `run.log_artifact()`. You can also track datasets in a remote filesystem (e.g. cloud storage in S3 or GCP) by reference, using a link or URI instead of the raw contents.\n2. **Version**: Define an artifact by giving it a type (`\"raw_data\"`, `\"preprocessed_data\"`, `\"balanced_data\"`) and a name (`\"imagenet_cats_10K\"`). When you log the same name again, W&B automatically creates a new version of the artifact with the latest contents.\n3. **Alias**: Set an alias like `\"best\"` or `\"production\"` to highlight the important versions in a lineage of artifacts.\n4. **Compare**: Select any two versions to browse the contents side-by-side. We're also working on a tool for dataset visualization, learn more here.\n5. **Download**: Obtain a local copy of the artifact or verify the contents by reference. \n\nFor more detail on these features, check out Artifacts how it works.", "matching_document_document_id": "7407e418-a665-438f-b605-d5dc12e4240e", "query_weight": 1.3006451612903225 }, { "query_id": "eec74c66-11de-4e88-bb90-40277d78fd24", "query": "Weights & Biases object detection", "matching_document": "' \n\n# Object Detection with PaddleDetection and Weights & Biases \n\nDescription: In this article, we'll learn how to use PaddleDetection to train a YOLOX model from scratch, logging all of your metrics and model checkpoints with Weights & Biases. \n\nBody: \n\n[PaddleDetection](https://github.com/PaddlePaddle/PaddleDetection) is an end-to-end object detection development kit based on [PaddlePaddle](https://github.com/PaddlePaddle/Paddle). It implements varied mainstream [object detection](http://wandb.ai/fully-connected/blog/object-detection), instance segmentation, tracking, and keypoint detection algorithms in modular design with configurable modules such as network components, data augmentations, and losses. \n\nPaddleDetection now comes with a built-in W&B integration that logs all your training and validation metrics, your model checkpoints, and their corresponding metadata. \n\n> \n\n# Setting Things Up \n\n## Installing the W&B SDK \n\n## Installing PaddleDetection \n\n## How to Download the COCO Dataset", "matching_document_document_id": "1f63ce30-b824-4a42-9a47-83ec6eea7b33", "query_weight": 0.1 }, { "query_id": "55430d7b-abba-4030-ace3-908400acc4b4", "query": "metric logging frequency", "matching_document": "## Limits & Performance\n### Best Practices for Fast Pages\n#### Logged Metrics\n##### Metric Frequency\nPick a logging frequency that is appropriate to the metric you are logging. As a general rule of thumb, the wider the metric the less frequently you should log it. Concretely, we recommend: \n\n* **Scalars**: <100,000 logged points per metric\n* **Media**: <50,000 logged points per metric\n* **Histograms**: <10,000 logged points per metric \n\n:::caution \n\n::: \n\n```\n# Training loop with 1m total steps\nfor step in range(1000000):\n# \u274c not recommended\nwandb.log(\n{\n\"scalar\": step, # 100,000 scalars\n\"media\": wandb.Image(...), # 100,000 images\n\"histogram\": wandb.Histogram(...), # 100,000 histograms\n}\n)\n\n# \u2705 recommended\nif step % 1000 == 0:\nwandb.log(\n{\n\"histogram\": wandb.Histogram(...), # 10,000 histograms\n},\ncommit=False,\n)\nif step % 200 == 0:\nwandb.log(\n{\n\"media\": wandb.Image(...), # 50,000 images\n},\ncommit=False,\n)\nif step % 100 == 0:\nwandb.log(\n{\n\"scalar\": step, # 100,000 scalars\n},\ncommit=True,\n) # Commit batched, per-step metrics together\n\n```", "matching_document_document_id": "1f22afa6-1f66-4a8c-bdd4-735b81f25e93", "query_weight": 0.5109677419354839 }, { "query_id": "589c50ae-3989-4157-920d-ea82d46da40d", "query": "How to correctly log images using WandbLogger in Python?", "matching_document": "## Introduction\n### Logging Images, Text, and More with WandbLogger\n\nPytorch Lightning is extensible through its callback system. That means we can create a custom callback to automatically log sample predictions during validation. \n\nWandbLogger provides convenient media logging functions: \n\n* `WandbLogger.log_text` for text data\n* `WandbLogger.log_image` for images\n* `WandbLogger.log_table` for . \n\nAn alternative to self.log in the model class is directly using wandb.log({dict}) or trainer.logger.experiment.log({dict}) \n\nBelow, we\u2019ll log the first 20 images in the first batch of the validation dataset along with the predicted and ground truth labels.", "matching_document_document_id": "c01b3eb7-5552-4587-a8ba-4f8c7f04ec54", "query_weight": 0.9754838709677419 }, { "query_id": "d5ea9312-3209-4908-b852-51853f741495", "query": "What is the structure of a sweep configuration in YAML or JSON?", "matching_document": "## Sweep configuration structure\n### Basic structure\ntrain.py\n```\nsweep\\_configuration = {\n\"name\": \"sweepdemo\",\n\"method\": \"bayes\",\n\"metric\": {\"goal\": \"minimize\", \"name\": \"validation\\_loss\"},\n\"parameters\": {\n\"learning\\_rate\": {\"min\": 0.0001, \"max\": 0.1},\n\"batch\\_size\": {\"values\": [16, 32, 64]},\n\"epochs\": {\"values\": [5, 10, 15]},\n\"optimizer\": {\"values\": [\"adam\", \"sgd\"]},\n},\n}\n\n``` \n\nDefine a sweep configuration in a YAML file if you want to manage sweeps interactively from the command line (CLI) \n\nconfig.yaml\n```\nprogram: train.py\nname: sweepdemo\nmethod: bayes\nmetric:\ngoal: minimize\nname: validation\\_loss\nparameters:\nlearning\\_rate:\nmin: 0.0001\nmax: 0.1\nbatch\\_size:\nvalues: [16, 32, 64]\nepochs:\nvalues: [5, 10, 15]\noptimizer:\nvalues: [\"adam\", \"sgd\"]\n\n```", "matching_document_document_id": "2a1ce862-7280-49c1-80c4-e4dfdd16914e", "query_weight": 0.7896774193548387 }, { "query_id": "ade63f49-346b-404b-b4e7-45cb5db0ff66", "query": "Weights & Biases code artifacts", "matching_document": "## Run\n### Methods\n#### `log_code`\n| Arguments | |\n| --- | --- |\n| `root` | The relative (to `os.getcwd()`) or absolute path to recursively find code from. |\n| `name` | (str, optional) The name of our code artifact. By default, we'll name the artifact `source-$PROJECT_ID-$ENTRYPOINT_RELPATH`. There may be scenarios where you want many runs to share the same artifact. Specifying name allows you to achieve that. |\n| `include_fn` | A callable that accepts a file path and (optionally) root path and returns True when it should be included and False otherwise. This defaults to: `lambda path, root: path.endswith(\".py\")` |\n| `exclude_fn` | A callable that accepts a file path and (optionally) root path and returns `True` when it should be excluded and `False` otherwise. This defaults to a function that excludes all files within `<root>/.wandb/` and `<root>/wandb/` directories. | \n\n#### Examples: \n\nBasic usage \n\n```\nrun.log\\_code()\n\n``` \n\nAdvanced usage", "matching_document_document_id": "1c1ab624-25b4-465c-ac9d-75d589ae2177", "query_weight": 0.5109677419354839 }, { "query_id": "76036718-ab6c-4803-aa48-7188a3ef3bcb", "query": "W&B sweep setup documentation", "matching_document": "\"\"\"Initialize a hyperparameter sweep.\n\n Search for hyperparameters that optimizes a cost function\n of a machine learning model by testing various combinations.\n\n Make note the unique identifier, `sweep_id`, that is returned.\n At a later step provide the `sweep_id` to a sweep agent.\n\n Args:\n sweep: The configuration of a hyperparameter search.\n (or configuration generator). See\n [Sweep configuration structure](https://docs.wandb.ai/guides/sweeps/define-sweep-configuration)\n for information on how to define your sweep.\n If you provide a callable, ensure that the callable does\n not take arguments and that it returns a dictionary that\n conforms to the W&B sweep config spec.\n entity: The username or team name where you want to send W&B\n runs created by the sweep to. Ensure that the entity you\n specify already exists. If you don't specify an entity,\n the run will be sent to your default entity,\n which is usually your username.\n project: The name of the project where W&B runs created from\n the sweep are sent to. If the project is not specified, the\n run is sent to a project labeled 'Uncategorized'.\n\n Returns:\n sweep_id: str. A unique identifier for the sweep.\n \"\"\"", "matching_document_document_id": "dde2c379-5417-4e62-9d30-034c0162fcc7", "query_weight": 0.1 }, { "query_id": "733253e4-cb4b-4131-b337-13dbf45a4983", "query": "Does Weights & Biases use S3 object versioning to manage artifact versions?", "matching_document": "## Data\n\n## AWS S3, Google Cloud Storage and W&B Artifacts \n\nWeights and Biases **Artifacts** enable you to log end-to-end training pipelines to ensure your experiments are always reproducible. \n\nData privacy is critical to Weights & Biases and so we support the creation of Artifacts from reference locations such as your own private cloud such as AWS S3 or Google Cloud Storage. Local, on-premises of W&B are also available upon request. \n\nBy default, W&B stores artifact files in a private Google Cloud Storage bucket located in the United States. All files are encrypted at rest and in transit. For sensitive files, we recommend a private W&B installation or the use of reference artifacts. \n\n##\u00a0Artifacts Reference Example\n**Create an artifact with the S3/GCS metadata** \n\nThe artifact only consists of metadata about the S3/GCS object such as its ETag, size, and version ID (if object versioning is enabled on the bucket). \n\n## Login to W&B \n\nLogin to Weights and Biases \n\n## Vehicle Loan Dataset", "matching_document_document_id": "15a6bbe9-03fd-4f70-89f9-3a0ad32cc614", "query_weight": 1.3006451612903225 }, { "query_id": "564759ad-1bea-40e5-b892-c47d7c6feba9", "query": "Data analysis using Weave in wandb", "matching_document": "## Weave\n### How it works\n1. First, install or update to the latest version of Weave with pip: \n\n```\npip install weave --upgrade\n\n```\n2. Load your dataframe into your notebook.\n3. View your dataframe with `weave.show`. \n\n```python title=\"weave.ipynb\" showLineNumbers\nimport weave\nfrom sklearn.datasets import load\\_iris\n\n# We load in the iris dataset for demonstrative purposes\n\niris = load\\_iris(as\\_frame=True)\ndf = iris.data.assign(target=iris.target\\_names[iris.target])\n\nweave.show(df)\n``` \n\nAn interactive weave dashboard will appear, similar to the animation shown below: \n\nThe following animations show how you can interactively plot charts and publish your dashboard to share with your colleagues: \n\n### Plot a chart \n\n1. Hover your mouse next to a panel and click **Add a new panel**.\n2. Copy the Weave Expression for the dataset you want to plot. This Weave Expression is the path/location of the dataset object in the Weave compute graph.\n3. Click on Table to change this Weave Panel type.\n4. From the dropdown, select **Plot**.", "matching_document_document_id": "ed920f49-e697-41bc-a8b4-201481542912", "query_weight": 0.1 }, { "query_id": "9f4855c4-8733-426f-9d44-edb049bccc69", "query": "Deleting artifacts in Weights & Biases", "matching_document": "## Delete artifacts\n:::note\nArtifacts that are scheduled for deletion with a TTL policy, deleted with the W&B SDK, or deleted with the W&B App UI are first soft-deleted. Artifacts that are soft deleted undergo garbage collection before they are hard-deleted.\n::: \n\n### Delete an artifact version \n\nTo delete an artifact version: \n\n1. Select the name of the artifact. This will expand the artifact view and list all the artifact versions associated with that artifact.\n2. From the list of artifacts, select the artifact version you want to delete.\n3. On the right hand side of the workspace, select the kebab dropdown.\n4. Choose Delete. \n\nAn artifact version can also be deleted programatically via the delete() method. See the examples below. \n\n### Delete multiple artifact versions with aliases \n\nThe following code example demonstrates how to delete artifacts that have aliases associated with them. Provide the entity, project name, and run ID that created the artifacts. \n\n### Delete multiple artifact versions with a specific alias", "matching_document_document_id": "f4d54671-aed9-4d97-96af-8780b8507acd", "query_weight": 0.9754838709677419 }, { "query_id": "4130184a-b0ac-47d9-8e09-b43eb8af2e44", "query": "wandb and Huggingface Transformers", "matching_document": "--- \n\n## displayed\\_sidebar: default \n\nimport { CTAButtons } from '@site/src/components/CTAButtons/CTAButtons.tsx'; \n\n# Hugging Face Transformers \n\nThe Hugging Face Transformers library makes state-of-the-art NLP models like BERT and training techniques like mixed precision and gradient checkpointing easy to use. The W&B integration adds rich, flexible experiment tracking and model versioning to interactive centralized dashboards without compromising that ease of use. \n\n## \ud83e\udd17 Next-level logging in few lines \n\n```\nos.environ[\"WANDB\\_PROJECT\"] = \"\" # name your W&B project\nos.environ[\"WANDB\\_LOG\\_MODEL\"] = \"checkpoint\" # log all model checkpoints\n\nfrom transformers import TrainingArguments, Trainer\n\nargs = TrainingArguments(..., report\\_to=\"wandb\") # turn on W&B logging\ntrainer = Trainer(..., args=args)\n\n``` \n\n:::info\nIf you'd rather dive straight into working code, check out this Google Colab.\n::: \n\n## Getting started: track experiments \n\n### 2) Name the project", "matching_document_document_id": "fe7a80f3-1c69-44cf-ab7c-fa24ac3e77dd", "query_weight": 1.1148387096774193 }, { "query_id": "96074556-e304-4080-b5fc-6368e55097bc", "query": "wandb API documentation", "matching_document": "## Api\n\nView source on GitHub \n\nUsed for querying the wandb server. \n\n```\nApi(\noverrides=None,\ntimeout: Optional[int] = None,\napi\\_key: Optional[str] = None\n) -> None\n\n``` \n\n#### Examples: \n\nMost common way to initialize \n\n```\n>>> wandb.Api()\n\n``` \n\n| Arguments | |\n| --- | --- |\n| `overrides` | (dict) You can set `base_url` if you are using a wandb server other than . You can also set defaults for `entity`, `project`, and `run`. | \n\n| Attributes | |\n| --- | --- |\n| | | \n\n## Methods \n\n### `artifact` \n\nView source \n\n```\nartifact(\nname, type=None\n)\n\n``` \n\nReturn a single artifact by parsing path in the form `entity/project/name`. \n\n| Arguments | |\n| --- | --- |\n| `name` | (str) An artifact name. May be prefixed with entity/project. Valid names can be in the following forms: name:version name:alias |\n| `type` | (str, optional) The type of artifact to fetch. | \n\n| Returns | |\n| --- | --- |\n| A `Artifact` object. | | \n\n### `artifact_type` \n\nView source \n\n### `flush`", "matching_document_document_id": "8d4ccf38-8646-42b5-abfa-b46e0c4ca9f5", "query_weight": 0.1 }, { "query_id": "2f5e5083-4f63-4d10-a3ef-e726ebb179a4", "query": "Case studies of Weights & Biases usage", "matching_document": "' \n\n# How to Use Weights & Biases for Experiment Tracking and Collaboration: A Case Study \n\nDescription: How a team of researchers use W&B to collaborate and track experiments \n\nBody: \n\nWe are a team of four friends pursuing our bachelor\u2019s degrees at ETH Zurich. This June we competed in the Student Cluster Competition at the International Supercomputing Conference 2019. Specifically, we represented ETH in the AI Task, which involved detecting extreme weather events such as cyclones in weather data. Weights and Biases was a useful tool for us, so we want to describe our workflow in developing and improving our model so other people can benefit from it as well. \n\n## Our basic workflow for using Weights & Biases: \n\n* Create a shared project on W&B\n* Set up W&B in our code with the API\n* Make a list of experiments to run\n* Automatically track experiments\n* Discuss, review results, and repeat steps 3 - 5 \n\n## Set Up \n\n## Experiment Tracking \n\n## Tips and Tricks", "matching_document_document_id": "1c36d097-6f04-4559-b1b8-22dab766606b", "query_weight": 0.7432258064516128 }, { "query_id": "c9eb76cc-e233-445c-a5b7-d04105e127e3", "query": "Weights & Biases Launch", "matching_document": "import Translate, {translate} from '@docusaurus/Translate';\nimport { CTAButtons } from '@site/src/components/CTAButtons/CTAButtons.tsx'; \n\n# Launch \n\nEasily scale training runs from your desktop to a compute resource like Amazon SageMaker, Kubernetes and more with W&B Launch. Once W&B Launch is configured, you can quickly run training scripts, model evaluation suites, prepare models for production inference, and more with a few clicks and commands. \n\n## How it works \n\nLaunch is composed of three fundamental components: **launch jobs**, **queues**, and **agents**. \n\nA *launch job* is a blueprint for configuring and running tasks in your ML workflow. Once you have a launch job, you can add it to a *launch queue*. A launch queue is a first-in, first-out (FIFO) queue where you can configure and submit your jobs to a particular compute target resource, such as Amazon SageMaker or a Kubernetes cluster. \n\n## How to get started", "matching_document_document_id": "a45f1b0e-ae06-43a3-bb95-d05039a60f0e", "query_weight": 0.9754838709677419 }, { "query_id": "108dc3fa-ff7c-483d-8c62-f95e778283c4", "query": "Weights & Biases project settings", "matching_document": "## wandb init\n\n**Usage** \n\n`wandb init [OPTIONS]` \n\n**Summary** \n\nConfigure a directory with Weights & Biases \n\n**Options** \n\n| **Option** | **Description** |\n| --- | --- |\n| -p, --project | The project to use. |\n| -e, --entity | The entity to scope the project to. |\n| --reset | Reset settings |\n| -m, --mode | Can be \"online\", \"offline\" or \"disabled\". Defaults to online. |", "matching_document_document_id": "ab01e901-8752-4cc2-84e5-9921b115a621", "query_weight": 0.1 }, { "query_id": "8f22c15f-e4dc-47f4-bbd8-198f8bffe528", "query": "Weights & Biases Huggingface Transformers", "matching_document": "## Language modeling\n### What Does Weights & Biases Have To Do With HuggingFace?\n\nWhile HuggingFace makes it straightforward to load and fine-tune models, Weights & Biases makes it easy to scale the volume and richness of your experiments. \n\nTo load and fine-tune a model with HuggingFace, it's recommended you use the provided HuggingFace trainer, which takes in your model, datasets, training arguments, and important metric computations: \n\n```\nfrom transformers import TrainingArguments, Trainer\n\n# Load model similar to what was shown above\n# model = AutoModelForTASK.from_pretrained(\"{SELECTED_MODEL}\")\n# (...)\ntraining_args = TrainingArguments(\noutput_dir=\"test_trainer\",\nevaluation_strategy=\"epoch\",\nreport_to=\"wandb\") #\ud83e\ude84\ud83d\udc1d\ntrainer = Trainer(\nmodel=model,\nargs=training_args,\ntrain_dataset=small_train_dataset,\neval_dataset=small_eval_dataset,\ncompute_metrics=compute_metrics,\n)\ntrainer.train()\n```", "matching_document_document_id": "149e66e9-e2e9-42b1-8a95-63af881f1451", "query_weight": 0.9754838709677419 }, { "query_id": "8bb5e0f1-6ca5-4a75-a4fd-0da5405c0395", "query": "does weights and balances have an integration with LangChain?", "matching_document": "## Prompts Quickstart\n### Use W&B Trace with LangChain\n#### 1. Set the LANGCHAIN\\_WANDB\\_TRACING environment variable\nFirst, set the LANGCHAIN\\_WANDB\\_TRACING environment variable to true. This will turn on automated Weights & Biases logging with LangChain: \n\n```\nimport os\n\n# turn on wandb logging for langchain\nos.environ[\"LANGCHAIN\\_WANDB\\_TRACING\"] = \"true\"\n\n``` \n\nThats it! Now any call to a LangChain LLM, Chain, Tool or Agent will be logged to Weights & Biases.", "matching_document_document_id": "6adad621-230b-42f0-badc-1738fef555a6", "query_weight": 1.2541935483870967 }, { "query_id": "0222bf07-dae6-4cf9-b2ec-f2c4d3441f9e", "query": "What is the syntax for logging model details in W&B?", "matching_document": "\"\"\"Logs a model artifact containing the contents inside the 'path' to a run and marks it as an output to this run.\n\n Arguments:\n path: (str) A path to the contents of this model,\n can be in the following forms:\n - `/local/directory`\n - `/local/directory/file.txt`\n - `s3://bucket/path`\n name: (str, optional) A name to assign to the model artifact that the file contents will be added to.\n The string must contain only the following alphanumeric characters: dashes, underscores, and dots.\n This will default to the basename of the path prepended with the current\n run id if not specified.\n aliases: (list, optional) Aliases to apply to the created model artifact,\n defaults to `[\"latest\"]`\n\n Examples:\n ```python\n run.log_model(\n path=\"/local/directory\",\n name=\"my_model_artifact\",\n aliases=[\"production\"],\n )\n ```", "matching_document_document_id": "4c6beb0c-1778-4132-a28c-a6f117299286", "query_weight": 1.2541935483870967 }, { "query_id": "f890fffe-4cfe-4b3e-af8c-a28de088c07a", "query": "data versioning best practices", "matching_document": "## Dataset Versioning\n### Visualize & easily share your data workflow\n* choose **meaningful organizational types** for your artifacts and jobs: for data, this could be `train`, `val`, or `test`; for scripts this could be `preprocess`, `train`, `evaluate`, etc. You may also want to log other data as artifacts: a model's predictions on fixed validation data, samples of generated output, evaluation metrics, etc.\n* **explore the artifact graph**: interact with all the connections between your code and data (input artifact(s) \u2192 script or job \u2192 output artifact(s)). Click \"explode\" on the compute graph to see all the versions for each artifact or all the runs of each script by job type. Click on individual nodes to see further details (file contents or code, annotations/metadata, config, timestamp, parent/child nodes, etc).\n* **iterate confidently**: all your experimental script runs and data will be saved and versioned automatically, so you can focus on the core modeling task and not worry about where and when you saved which version of your dataset or code\n* **share & replicate easily**: once you've integrated artifacts, you and your teammates can smoothly rerun the same workflow and pull from identical datasets (defaulting to the latest/best version), even to train in a different context/on different hardware", "matching_document_document_id": "f45869fa-ad3d-428e-86cd-94784639cb4f", "query_weight": 1.904516129032258 }, { "query_id": "f359a38c-16e8-4a36-aac3-a7b5b87e74fe", "query": "wandb.init resume allow vs auto behavior", "matching_document": "| `resume` | (bool, str, optional) Sets the resuming behavior. Options: `\"allow\"`, `\"must\"`, `\"never\"`, `\"auto\"` or `None`. Defaults to `None`. Cases: - `None` (default): If the new run has the same ID as a previous run, this run overwrites that data. - `\"auto\"` (or `True`): if the previous run on this machine crashed, automatically resume it. Otherwise, start a new run. - `\"allow\"`: if id is set with `init(id=\"UNIQUE_ID\")` or `WANDB_RUN_ID=\"UNIQUE_ID\"` and it is identical to a previous run, wandb will automatically resume the run with that id. Otherwise, wandb will start a new run. - `\"never\"`: if id is set with `init(id=\"UNIQUE_ID\")` or `WANDB_RUN_ID=\"UNIQUE_ID\"` and it is identical to a previous run, wandb will crash. - `\"must\"`: if id is set with `init(id=\"UNIQUE_ID\")` or `WANDB_RUN_ID=\"UNIQUE_ID\"` and it is identical to a previous run, wandb will automatically resume the run with the id. Otherwise, wandb will crash. See our guide to resuming runs for more. |\n| `reinit` | (bool, optional) Allow multiple `wandb.init()` calls in the same process. (default: `False`) |\n| `magic` | (bool, dict, or str, optional) The bool controls whether we try to auto-instrument your script, capturing basic details of your run without you having to add more wandb code. (default: `False`) You can also pass a dict, json string, or yaml filename. |\n| `config_exclude_keys` | (list, optional) string keys to exclude from `wandb.config`. |\n| `config_include_keys` | (list, optional) string keys to include in `wandb.config`. |", "matching_document_document_id": "01c9bdf0-fd26-4350-8de1-e75588077941", "query_weight": 1.0219354838709678 }, { "query_id": "d76660b5-c30c-454f-ae39-798c057c053e", "query": "sweep configuration", "matching_document": "## Sweep configuration structure\n### Basic structure\ntrain.py\n```\nsweep\\_configuration = {\n\"name\": \"sweepdemo\",\n\"method\": \"bayes\",\n\"metric\": {\"goal\": \"minimize\", \"name\": \"validation\\_loss\"},\n\"parameters\": {\n\"learning\\_rate\": {\"min\": 0.0001, \"max\": 0.1},\n\"batch\\_size\": {\"values\": [16, 32, 64]},\n\"epochs\": {\"values\": [5, 10, 15]},\n\"optimizer\": {\"values\": [\"adam\", \"sgd\"]},\n},\n}\n\n``` \n\nDefine a sweep configuration in a YAML file if you want to manage sweeps interactively from the command line (CLI) \n\nconfig.yaml\n```\nprogram: train.py\nname: sweepdemo\nmethod: bayes\nmetric:\ngoal: minimize\nname: validation\\_loss\nparameters:\nlearning\\_rate:\nmin: 0.0001\nmax: 0.1\nbatch\\_size:\nvalues: [16, 32, 64]\nepochs:\nvalues: [5, 10, 15]\noptimizer:\nvalues: [\"adam\", \"sgd\"]\n\n```", "matching_document_document_id": "2a1ce862-7280-49c1-80c4-e4dfdd16914e", "query_weight": 0.7896774193548387 }, { "query_id": "f5f0f0e6-adba-4e09-b7c6-2ceb777e8df7", "query": "Examples of molecular data logging in W&B", "matching_document": "# Log points and boxes in W&B\npoint\\_scene = wandb.Object3D(\n{\n\"type\": \"lidar/beta\",\n\"points\": np.array( # add points, as in a point cloud\n[[0.4, 1, 1.3], [1, 1, 1], [1.2, 1, 1.2]]\n),\n\"boxes\": np.array( # draw 3d boxes\n[\n{\n\"corners\": [\n[0, 0, 0],\n[0, 1, 0],\n[0, 0, 1],\n[1, 0, 0],\n[1, 1, 0],\n[0, 1, 1],\n[1, 0, 1],\n[1, 1, 1],\n],\n\"label\": \"Box\",\n\"color\": [123, 321, 111],\n},\n{\n\"corners\": [\n[0, 0, 0],\n[0, 2, 0],\n[0, 0, 2],\n[2, 0, 0],\n[2, 2, 0],\n[0, 2, 2],\n[2, 0, 2],\n[2, 2, 2],\n],\n\"label\": \"Box-2\",\n\"color\": [111, 321, 0],\n},\n]\n),\n\"vectors\": np.array( # add 3d vectors\n[{\"start\": [0, 0, 0], \"end\": [0.1, 0.2, 0.5]}]\n),\n}\n)\nwandb.log({\"point\\_scene\": point\\_scene})\n\n``` \n\n```\nwandb.log({\"protein\": wandb.Molecule(\"6lu7.pdb\")})\n\n``` \n\nLog molecular data in any of 10 file types:`pdb`, `pqr`, `mmcif`, `mcif`, `cif`, `sdf`, `sd`, `gro`, `mol2`, or `mmtf.` \n\nW&B also supports logging molecular data from SMILES strings, `rdkit` `mol` files, and `rdkit.Chem.rdchem.Mol` objects.", "matching_document_document_id": "b701ff5d-ee83-4f78-94ff-bab9439bdf4b", "query_weight": 1.2541935483870967 }, { "query_id": "d99a02dc-7416-4653-a745-46892a727314", "query": "LangChain API integration with Weights & Biases", "matching_document": "## Prompts Quickstart\n### Use W&B Trace with LangChain\n#### 1. Set the LANGCHAIN\\_WANDB\\_TRACING environment variable\nFirst, set the LANGCHAIN\\_WANDB\\_TRACING environment variable to true. This will turn on automated Weights & Biases logging with LangChain: \n\n```\nimport os\n\n# turn on wandb logging for langchain\nos.environ[\"LANGCHAIN\\_WANDB\\_TRACING\"] = \"true\"\n\n``` \n\nThats it! Now any call to a LangChain LLM, Chain, Tool or Agent will be logged to Weights & Biases.", "matching_document_document_id": "6adad621-230b-42f0-badc-1738fef555a6", "query_weight": 1.2541935483870967 }, { "query_id": "8bd67c24-3ede-4f4c-8f87-1671c83b4df5", "query": "Weights & Biases introduction", "matching_document": "## Weights & Biases overview\nW&B is a platform that helps data scientists track their models, datasets, system information and more. With a few lines of code, you can start tracking everything about these features. It's free for personal use. Team use is normally a paid utility, but teams for academic purposes are free. You can use W&B with your favourite framework, like TensorFlow, Keras, PyTorch, Sklearn, fastai and many others. \n\nAll tracking information is sent to a dedicated project page on the W&B UI, where you can open high quality visualizations, aggregate information and compare models or parameters. One of the advantages of remotely storing the experiment\u2019s information is that it is easy to collaborate on the same project and share the results with your teammates. \n\nW&B provides 4 useful tools: \n\n* Dashboard: Experiment tracking\n* Artifacts: Dataset versioning, model versioning\n* Sweeps: Hyperparameter optimization\n* Reports: Save and share reproducible findings", "matching_document_document_id": "8c1600ad-9b6e-45cd-86ae-03afd8f80332", "query_weight": 0.9754838709677419 }, { "query_id": "0ffaf580-5fae-42a9-8f72-159b60b1c388", "query": "how to use wandb", "matching_document": "## Quickstart\n\nInstall W&B and start tracking your machine learning experiments in minutes. \n\n## 1. Create an account and install W&B \n\nBefore you get started, make sure you create an account and install W&B: \n\n1. Sign up for a free account at and then log in to your wandb account.\n2. Install the wandb library on your machine in a Python 3 environment using `pip`. \n\nThe following code snippets demonstrate how to install and log into W&B using the W&B CLI and Python Library: \n\nInstall the CLI and Python library for interacting with the Weights and Biases API: \n\n```\npip install wandb\n\n``` \n\nInstall the CLI and Python library for interacting with the Weights and Biases API: \n\n```\n!pip install wandb\n\n``` \n\n## 2. Log in to W&B \n\nNext, log in to W&B: \n\n```\nwandb login\n\n``` \n\nOr if you are using W&B Server (including **Dedicated Cloud** or **Self-managed**): \n\n```\nwandb login --relogin --host=http://your-shared-local-host.com\n\n``` \n\n## 3. Start a run and track hyperparameters", "matching_document_document_id": "4c3d69bd-11c2-4856-87e5-bcc4fa96d3ab", "query_weight": 0.9290322580645161 }, { "query_id": "b45751fe-d5e7-48ec-ba30-8358dc2ee76e", "query": "Step by step guide to start with WandB", "matching_document": "## Quickstart\n\nInstall W&B and start tracking your machine learning experiments in minutes. \n\n## 1. Create an account and install W&B \n\nBefore you get started, make sure you create an account and install W&B: \n\n1. Sign up for a free account at and then log in to your wandb account.\n2. Install the wandb library on your machine in a Python 3 environment using `pip`. \n\nThe following code snippets demonstrate how to install and log into W&B using the W&B CLI and Python Library: \n\nInstall the CLI and Python library for interacting with the Weights and Biases API: \n\n```\npip install wandb\n\n``` \n\nInstall the CLI and Python library for interacting with the Weights and Biases API: \n\n```\n!pip install wandb\n\n``` \n\n## 2. Log in to W&B \n\nNext, log in to W&B: \n\n```\nwandb login\n\n``` \n\nOr if you are using W&B Server (including **Dedicated Cloud** or **Self-managed**): \n\n```\nwandb login --relogin --host=http://your-shared-local-host.com\n\n``` \n\n## 3. Start a run and track hyperparameters", "matching_document_document_id": "4c3d69bd-11c2-4856-87e5-bcc4fa96d3ab", "query_weight": 0.9290322580645161 }, { "query_id": "ddd5273c-31c5-439c-9430-b7c625c3c664", "query": "LangChain compatibility with Weights & Biases", "matching_document": "## Prompts Quickstart\n### Use W&B Trace with LangChain\n#### 1. Set the LANGCHAIN\\_WANDB\\_TRACING environment variable\nFirst, set the LANGCHAIN\\_WANDB\\_TRACING environment variable to true. This will turn on automated Weights & Biases logging with LangChain: \n\n```\nimport os\n\n# turn on wandb logging for langchain\nos.environ[\"LANGCHAIN\\_WANDB\\_TRACING\"] = \"true\"\n\n``` \n\nThats it! Now any call to a LangChain LLM, Chain, Tool or Agent will be logged to Weights & Biases.", "matching_document_document_id": "6adad621-230b-42f0-badc-1738fef555a6", "query_weight": 1.2541935483870967 }, { "query_id": "b372408f-87a1-45df-ae99-752fb66d1030", "query": "SageMaker IAM roles", "matching_document": "## Set up for SageMaker\n### Prerequisites\n1. **Setup SageMaker in your AWS account.** See the SageMaker Developer guide for more information.\n2. **Create an Amazon ECR repository** to store images you want to execute on Amazon SageMaker. See the Amazon ECR documentation for more information.\n3. **Create an Amazon S3 buckets** to store SageMaker inputs and outputs for your SageMaker training jobs. See the Amazon S3 documentation for more information. Make note of the S3 bucket URI and directory.\n4. **Create IAM execution role.** The role used in the SageMaker training job requires the following permissions to work. These permissions allow for logging events, pulling from ECR, and interacting with input and output buckets. (Note: if you already have this role for SageMaker training jobs, you do not need to create it again.)\nIAM role policy\n```\n{\n\"Version\": \"2012-10-17\",\n\"Statement\": [\n{\n\"Effect\": \"Allow\",\n\"Action\": [\n\"cloudwatch:PutMetricData\",\n\"logs:CreateLogStream\",\n\"logs:PutLogEvents\",\n\"logs:CreateLogGroup\",\n\"logs:DescribeLogStreams\",\n\"ecr:GetAuthorizationToken\"\n],\n\"Resource\": \"\\*\"\n},\n{\n\"Effect\": \"Allow\",\n\"Action\": [\n\"s3:ListBucket\"\n],\n\"Resource\": [\n\"arn:aws:s3:::\"\n]\n},\n{\n\"Effect\": \"Allow\",\n\"Action\": [\n\"s3:GetObject\",\n\"s3:PutObject\"\n],\n\"Resource\": [\n\"arn:aws:s3:::/\",\n\"arn:aws:s3:::/\"\n]\n},\n{\n\"Effect\": \"Allow\",\n\"Action\": [\n\"ecr:BatchCheckLayerAvailability\",\n\"ecr:GetDownloadUrlForLayer\",\n\"ecr:BatchGetImage\"\n],\n\"Resource\": \"arn:aws:ecr:::repository/\"\n}\n]\n}\n\n```", "matching_document_document_id": "455be611-540d-4ed2-a34d-bd233f9e98d6", "query_weight": 1.0683870967741935 }, { "query_id": "a3366b20-36e2-4f56-9678-fc14be88a043", "query": "W&B API keys", "matching_document": "## Dagster\n### Before you get started\n#### How to get an API key\n1. Log in to W&B. Note: if you are using W&B Server ask your admin for the instance host name.\n2. Collect your API key by navigating to the authorize page or in your user/team settings. For a production environment we recommend using a service account to own that key.\n3. Set an environment variable for that API key export `WANDB_API_KEY=YOUR_KEY`. \n\nThe proceeding examples demonstrate where to specify your API key in your Dagster code. Make sure to specify your entity and project name within the `wandb_config` nested dictionary. You can pass different `wandb_config` values to different ops/assets if you want to use a different W&B Project. For more information about possible keys you can pass, see the Configuration section below. \n\nExample: configuration for `@job`", "matching_document_document_id": "ed58e334-dcd9-4d21-998b-18209f34c3a0", "query_weight": 0.1 }, { "query_id": "7f2523d7-12c0-4d53-8aea-d887a7a82a83", "query": "weights & biases table features", "matching_document": "## Visualizing the Results Interactively as a W&B Table\n\nW&B Tables is an awesome Weights & Biases feature that lets interactively visualize and explore tabular data. And it is extremely easy to create one. \n\nAll we need to do is define a Pandas DataFrame (basically, defining a table in Python) with the four relevant to us columns. \n\n> \n\n```\nimport pandas as pd\n\npositive = predictions[:, 0].tolist()\nnegative = predictions[:, 1].tolist()\nneutral = predictions[:, 2].tolist()\n\ntable = {'Headline':headlines_list,\n\"Positive\":positive,\n\"Negative\":negative,\n\"Neutral\":neutral}\n\ndf = pd.DataFrame(table, columns = [\"Headline\", \"Positive\", \"Negative\", \"Neutral\"])\n\ndf.head(5)\n``` \n\n### Logging a W&B Table \n\nOkay. We're just 5 lines of code away from logging a W&B Table! The first thing we do is pip install and import the W&B library. \n\n```\n!pip install wandb\nimport wandb\n```", "matching_document_document_id": "5d634218-ff45-45d4-93ee-1495af0fa478", "query_weight": 0.9754838709677419 }, { "query_id": "5db826f2-fbf6-4bb6-91ba-cc99cbe74305", "query": "User guide for managing W&B experiment runs", "matching_document": "## Create an Experiment\n### Best Practices\n\nThe following are some suggested guidelines to consider when you create experiments: \n\n1. **Config**: Track hyperparameters, architecture, dataset, and anything else you'd like to use to reproduce your model. These will show up in columns\u2014 use config columns to group, sort, and filter runs dynamically in the app.\n2. **Project**: A project is a set of experiments you can compare together. Each project gets a dedicated dashboard page, and you can easily turn on and off different groups of runs to compare different model versions.\n3. **Notes**: A quick commit message to yourself. The note can be set from your script. You can edit notes at a later time on the Overview section of your project's dashboard on the W&B App.\n4. **Tags**: Identify baseline runs and favorite runs. You can filter runs using tags. You can edit tags at a later time on the Overview section of your project's dashboard on the W&B App.", "matching_document_document_id": "fba62bc2-107b-4676-9e4a-ba03137729e2", "query_weight": 0.1 }, { "query_id": "716583a7-bf5c-40ff-a4b4-728ee7998af1", "query": "artifact versioning", "matching_document": "## Create new artifact versions\n### Create new artifact versions from scratch\n* **Single run**: A single run provides all the data for a new version. This is the most common case and is best suited when the run fully recreates the needed data. For example: outputting saved models or model predictions in a table for analysis.\n* **Distributed runs**: A set of runs collectively provides all the data for a new version. This is best suited for distributed jobs which have multiple runs generating data, often in parallel. For example: evaluating a model in a distributed manner, and outputting the predictions. \n\nW&B will create a new artifact and assign it a `v0` alias if you pass a name to the `wandb.Artifact` API that does not exist in your project. W&B checksums the contents when you log again to the same artifact. If the artifact changed, W&B saves a new version `v1`. \n\nW&B will retrieve an existing artifact if you pass a name and artifact type to the `wandb.Artifact` API that matches an existing artifact in your project. The retrieved artifact will have a version greater than 1.", "matching_document_document_id": "2e896de8-7ee0-41ba-983c-2ca7b0b7cb6e", "query_weight": 1.904516129032258 }, { "query_id": "98492009-a299-4285-bc82-8e0644bc6268", "query": "connecting Weights & Biases with LangChain", "matching_document": "## Prompts Quickstart\n### Use W&B Trace with LangChain\n#### 1. Set the LANGCHAIN\\_WANDB\\_TRACING environment variable\nFirst, set the LANGCHAIN\\_WANDB\\_TRACING environment variable to true. This will turn on automated Weights & Biases logging with LangChain: \n\n```\nimport os\n\n# turn on wandb logging for langchain\nos.environ[\"LANGCHAIN\\_WANDB\\_TRACING\"] = \"true\"\n\n``` \n\nThats it! Now any call to a LangChain LLM, Chain, Tool or Agent will be logged to Weights & Biases.", "matching_document_document_id": "6adad621-230b-42f0-badc-1738fef555a6", "query_weight": 1.2541935483870967 }, { "query_id": "c68c7936-1fbb-4279-b3f6-aeea237b4f60", "query": "Weights & Biases custom line chart", "matching_document": "' \n\n# Creating Custom Line Plots With Weights & Biases \n\nDescription: This article provides usage and examples for wandb.plot.line(), explaining how to log a custom line plot natively in just a few lines. \n\nBody: \n\n# Method: wandb.plot.line() \n\nLog a custom line plot\u2014a list of connected/ordered points (x, y) on a pair of arbitrary axes x and y\u2014natively in a few lines: \n\n```\ndata = [[x, y] for (x, y) in zip(x_values, y_values)]\ntable = wandb.Table(data=data, columns = [\"x\", \"y\"])\nwandb.log({\"my_custom_plot_id\" : wandb.plot.line(table,\n\"x\", \"y\", title=\"Custom Y vs X Line Plot\")})\n``` \n\nYou can use this to log curves on any two dimensions. Note that if you're plotting two lists of values against each other, the number of values in the lists must match exactly (i.e. each point must have an x and a y). \n\n# Basic Usage Example \n\n# Customized Usage \n\nThere are many ways to customize the line plot using the [Vega visualization grammar](https://vega.github.io/). \n\nHere are some simple ones:", "matching_document_document_id": "8727d4a1-a537-4f65-a154-c1cb409f4c3b", "query_weight": 0.1 }, { "query_id": "0985226b-7356-4de0-a8a1-e1f5c3d964d0", "query": "Best practices for beginners in wandb", "matching_document": "## \u270d\ufe0f W&B Best Practices\n\n1. **Projects**: Log multiple runs to a project to compare them. `wandb.init(project=\"project-name\")`\n2. **Groups**: For multiple processes or cross validation folds, log each process as a runs and group them together. `wandb.init(group=\"experiment-1\")`\n3. **Tags**: Add tags to track your current baseline or production model.\n4. **Notes**: Type notes in the table to track the changes between runs.\n5. **Reports**: Take quick notes on progress to share with colleagues and make dashboards and snapshots of your ML projects.", "matching_document_document_id": "aa2dbfa2-471e-4ec7-b1a2-60a608aa6873", "query_weight": 0.9290322580645161 }, { "query_id": "485369f3-fd2f-4f95-aa66-3e69096c6bc9", "query": "Integrating large language models with Weights & Biases", "matching_document": "## OpenAI API\n\nUse the Weights & Biases OpenAI API integration to log requests, responses, token counts and model metadata with 1 line of code for all OpenAI models, including fine-tuned models. \n\n:::info\nThe W&B autolog integration works with `openai <= 0.28.1`. Install the correct version of `openai` with `pip install openai==0.28.1`.\n::: \n\n**Try in a Colab Notebook here \u2192** \n\nWith just 1 line of code you can now automatically log inputs and outputs from the OpenAI Python SDK to Weights & Biases! \n\nOnce you start logging your API inputs and outputs you can quickly evaluate the performance of difference prompts, compare different model settings (such as temperature), and track other usage metrics such as token usage. \n\nTo get started, pip install the `wandb` library, then follow the steps below: \n\n### 1. Import autolog and initialise it \n\nFirst, import `autolog` from `wandb.integration.openai` and initialise it. \n\n### 2. Call the OpenAI API \n\n### 3. View your OpenAI API inputs and responses", "matching_document_document_id": "04695bcc-0c96-4e1e-baf7-113b1b4dca09", "query_weight": 0.9754838709677419 }, { "query_id": "cf000873-c3d6-45b2-b9a4-99f924167701", "query": "Weights & Biases markdown support", "matching_document": "## Edit a report\n#### Markdown\nAdd markdown to your report interactively with the App UI or with the W&B SDK. \n\nEnter a forward slash (`/`) in the report to display a dropdown menu. From the dropdown choose **Markdown**. \n\nUse the `wandb.apis.reports.MarkdownBlock` Class to create a markdown block programmatically. Pass a string to the `text` parameter: \n\n```\nimport wandb\nimport wandb.apis.reports as wr\n\nreport = wr.Report(project=\"report-editing\")\n\nreport.blocks = [\nwr.MarkdownBlock(text=\"Markdown cell with \\*italics\\* and \\*\\*bold\\*\\* and $e=mc^2$\")\n]\n\n``` \n\nThis will render a markdown block similar to:", "matching_document_document_id": "c651c051-93c3-41b2-bfc8-844d8e467a7f", "query_weight": 0.1 }, { "query_id": "00d2b3de-7160-4c75-acc5-33552d2f40a8", "query": "Artifact versioning best practices in Weights & Biases", "matching_document": "## Create new artifact versions\n### Create new artifact versions from scratch\n* **Single run**: A single run provides all the data for a new version. This is the most common case and is best suited when the run fully recreates the needed data. For example: outputting saved models or model predictions in a table for analysis.\n* **Distributed runs**: A set of runs collectively provides all the data for a new version. This is best suited for distributed jobs which have multiple runs generating data, often in parallel. For example: evaluating a model in a distributed manner, and outputting the predictions. \n\nW&B will create a new artifact and assign it a `v0` alias if you pass a name to the `wandb.Artifact` API that does not exist in your project. W&B checksums the contents when you log again to the same artifact. If the artifact changed, W&B saves a new version `v1`. \n\nW&B will retrieve an existing artifact if you pass a name and artifact type to the `wandb.Artifact` API that matches an existing artifact in your project. The retrieved artifact will have a version greater than 1.", "matching_document_document_id": "2e896de8-7ee0-41ba-983c-2ca7b0b7cb6e", "query_weight": 1.3006451612903225 }, { "query_id": "05172110-28c2-468f-a27c-205ae8be763f", "query": "What is the process for exporting chart data using the Weights & Biases API?", "matching_document": "## Import & Export Data\n### Export Data\nUse the Public API to export or update data that you have saved to W&B. Before using this API, you'll want to log data from your script \u2014 check the Quickstart for more details. \n\n**Use Cases for the Public API** \n\n* **Export Data**: Pull down a dataframe for custom analysis in a Jupyter Notebook. Once you have explored the data, you can sync your findings by creating a new analysis run and logging results, for example: `wandb.init(job_type=\"analysis\")`\n* **Update Existing Runs**: You can update the data logged in association with a W&B run. For example, you might want to update the config of a set of runs to include additional information, like the architecture or a hyperparameter that wasn't originally logged. \n\nSee the Generated Reference Docs for details on available functions. \n\n### Authentication \n\nAuthenticate your machine with your API key in one of two ways: \n\n1. Run `wandb login` on the command line and paste in your API key.\n2. Set the `WANDB_API_KEY` environment variable to your API key.", "matching_document_document_id": "6fab5684-679d-4e4c-baa9-1016ff28e256", "query_weight": 1.8116129032258064 }, { "query_id": "11eb25ec-5f09-41bd-b8ca-572a4b3f93f1", "query": "Examples of data analysis with W&B tables in Python", "matching_document": "## Data\n### Get Train/Validation Split\n#### Log Dataset with W&B Tables\n\nWith W&B Tables you can log, query, and analyze tabular data that contains rich media such as images, video, audio and more. With it you can understand your datasets, visualize model predictions, and share insights, for more see more in our W&B Tables Guide \n\n```\n# Create a wandb run, with an optional \"log-dataset\" job type to keep things tidy\nrun = wandb.init(project=WANDB_PROJECT, job_type='log-dataset') # config is optional here\n\n# Create a W&B Table and log 1000 random rows of the dataset to explore\ntable = wandb.Table(dataframe=trndat.sample(1000))\n\n# Log the Table to your W&B workspace\nwandb.log({'processed_dataset': table})\n\n# Close the wandb run\nwandb.finish()\n\n```", "matching_document_document_id": "3ecbb57c-82f5-4aba-9abc-b79758c44084", "query_weight": 1.2541935483870967 }, { "query_id": "509bd820-8c1d-47c3-a36a-cdb0f0e44bdc", "query": "How to log and visualize a list of metrics in Weights & Biases?", "matching_document": "## Visualizing History\nWeights & Biases makes it super easy to visualize history automatically. You can simply add a WandbCallback to your model.fit() to automatically save all the metrics and the loss values tracked. Check out the [docs](https://docs.wandb.com/library/frameworks/keras) for more info. \n\n```\nimport wandb\nfrom wandb.keras import WandbCallback\n\n# 1. Start a new run\nwandb.init(project=\"gpt-3\")\n\u200d\n# ... Define a model\n\n# 2. Log layer dimensions and metrics over time\nmodel.fit(X_train, y_train, validation_data=(X_test, y_test),\ncallbacks=[WandbCallback()])\n``` \n\nIn the plots below, you can see all the metrics collected by the history object visualized by Weights & Biases with just a couple lines of code: \n\n## Weights & Biases \n\nWeights & Biases helps you keep track of your machine learning experiments. Use our tool to log hyperparameters and output metrics from your runs, then visualize and compare results and quickly share findings with your colleagues.", "matching_document_document_id": "61c8f9c1-41d4-41ad-b640-d1a84e7ab234", "query_weight": 0.1 }, { "query_id": "a1fb3334-b10b-47db-bfff-165c29163572", "query": "Wandb image logging best practices", "matching_document": "## Limits & Performance\n### Best Practices for Fast Pages\n#### Logged Metrics\n##### Metric Frequency\nPick a logging frequency that is appropriate to the metric you are logging. As a general rule of thumb, the wider the metric the less frequently you should log it. Concretely, we recommend: \n\n* **Scalars**: <100,000 logged points per metric\n* **Media**: <50,000 logged points per metric\n* **Histograms**: <10,000 logged points per metric \n\n:::caution \n\n::: \n\n```\n# Training loop with 1m total steps\nfor step in range(1000000):\n# \u274c not recommended\nwandb.log(\n{\n\"scalar\": step, # 100,000 scalars\n\"media\": wandb.Image(...), # 100,000 images\n\"histogram\": wandb.Histogram(...), # 100,000 histograms\n}\n)\n\n# \u2705 recommended\nif step % 1000 == 0:\nwandb.log(\n{\n\"histogram\": wandb.Histogram(...), # 10,000 histograms\n},\ncommit=False,\n)\nif step % 200 == 0:\nwandb.log(\n{\n\"media\": wandb.Image(...), # 50,000 images\n},\ncommit=False,\n)\nif step % 100 == 0:\nwandb.log(\n{\n\"scalar\": step, # 100,000 scalars\n},\ncommit=True,\n) # Commit batched, per-step metrics together\n\n```", "matching_document_document_id": "1f22afa6-1f66-4a8c-bdd4-735b81f25e93", "query_weight": 0.9754838709677419 }, { "query_id": "49603fcc-231b-4d54-b525-aed5a7eb673b", "query": "Is there an official integration between Weights & Biases and LangChain?", "matching_document": "## Prompts Quickstart\n### Use W&B Trace with LangChain\n#### 1. Set the LANGCHAIN\\_WANDB\\_TRACING environment variable\nFirst, set the LANGCHAIN\\_WANDB\\_TRACING environment variable to true. This will turn on automated Weights & Biases logging with LangChain: \n\n```\nimport os\n\n# turn on wandb logging for langchain\nos.environ[\"LANGCHAIN\\_WANDB\\_TRACING\"] = \"true\"\n\n``` \n\nThats it! Now any call to a LangChain LLM, Chain, Tool or Agent will be logged to Weights & Biases.", "matching_document_document_id": "6adad621-230b-42f0-badc-1738fef555a6", "query_weight": 1.2541935483870967 }, { "query_id": "c213b5d5-aa9e-4d49-b3d2-2b2e0aa1d558", "query": "Customizing chart views in Weights & Biases for better visualization", "matching_document": "## Custom Charts Walkthrough\n### 3. Customize the chart\nNow that looks pretty good, but I'd like to switch from a scatter plot to a line plot. Click **Edit** to change the Vega spec for this built in chart. Follow along in this workspace. \n\nI updated the Vega spec to customize the visualization: \n\n* add titles for the plot, legend, x-axis, and y-axis (set \u201ctitle\u201d for each field)\n* change the value of \u201cmark\u201d from \u201cpoint\u201d to \u201cline\u201d\n* remove the unused \u201csize\u201d field \n\nTo save this as a preset that you can use elsewhere in this project, click **Save as** at the top of the page. Here's what the result looks like, along with an ROC curve:", "matching_document_document_id": "2e1d68d2-2ae7-4aaa-af4a-9121f37d5d63", "query_weight": 1.8116129032258064 }, { "query_id": "ac2e639d-1180-48fb-8d01-899b03da291e", "query": "logging metrics best practices", "matching_document": "## Limits & Performance\n### Best Practices for Fast Pages\n#### Logged Metrics\n##### Metric Frequency\nPick a logging frequency that is appropriate to the metric you are logging. As a general rule of thumb, the wider the metric the less frequently you should log it. Concretely, we recommend: \n\n* **Scalars**: <100,000 logged points per metric\n* **Media**: <50,000 logged points per metric\n* **Histograms**: <10,000 logged points per metric \n\n:::caution \n\n::: \n\n```\n# Training loop with 1m total steps\nfor step in range(1000000):\n# \u274c not recommended\nwandb.log(\n{\n\"scalar\": step, # 100,000 scalars\n\"media\": wandb.Image(...), # 100,000 images\n\"histogram\": wandb.Histogram(...), # 100,000 histograms\n}\n)\n\n# \u2705 recommended\nif step % 1000 == 0:\nwandb.log(\n{\n\"histogram\": wandb.Histogram(...), # 10,000 histograms\n},\ncommit=False,\n)\nif step % 200 == 0:\nwandb.log(\n{\n\"media\": wandb.Image(...), # 50,000 images\n},\ncommit=False,\n)\nif step % 100 == 0:\nwandb.log(\n{\n\"scalar\": step, # 100,000 scalars\n},\ncommit=True,\n) # Commit batched, per-step metrics together\n\n```", "matching_document_document_id": "1f22afa6-1f66-4a8c-bdd4-735b81f25e93", "query_weight": 0.5109677419354839 }, { "query_id": "118b9cea-5496-4f85-842a-887e565bc258", "query": "wandb.finish() usage in training scripts", "matching_document": "## Hugging Face Transformers\n### Getting started: track experiments\n#### 6) Finish your W&B Run (Notebook only)\n\nIf your training is encapsulated in a Python script, the W&B run will end when your script finishes. \n\nIf you are using a Jupyter or Google Colab notebook, you'll need to tell us when you're done with training by calling `wandb.finish()`. \n\n```\ntrainer.train() # start training and logging to W&B\n\n# post-training analysis, testing, other logged code\n\nwandb.finish()\n\n```", "matching_document_document_id": "6edbb7d2-323e-49de-9ca2-f289602fbef0", "query_weight": 0.6967741935483871 }, { "query_id": "d8b3531f-44dc-42cb-9af4-d16038fb8dc2", "query": "How does 'auto' setting for 'resume' in 'wandb.init' differ from 'allow'?", "matching_document": "metadata will be stored. When you call `download()` on an artifact,\n this is the directory where downloaded files will be saved. By default,\n this is the `./wandb` directory.\n resume: (bool, str, optional) Sets the resuming behavior. Options:\n `\"allow\"`, `\"must\"`, `\"never\"`, `\"auto\"` or `None`. Defaults to `None`.\n Cases:\n - `None` (default): If the new run has the same ID as a previous run,\n this run overwrites that data.\n - `\"auto\"` (or `True`): if the previous run on this machine crashed,\n automatically resume it. Otherwise, start a new run.\n - `\"allow\"`: if id is set with `init(id=\"UNIQUE_ID\")` or\n `WANDB_RUN_ID=\"UNIQUE_ID\"` and it is identical to a previous run,\n wandb will automatically resume the run with that id. Otherwise,\n wandb will start a new run.\n - `\"never\"`: if id is set with `init(id=\"UNIQUE_ID\")` or\n `WANDB_RUN_ID=\"UNIQUE_ID\"` and it is identical to a previous run,\n wandb will crash.", "matching_document_document_id": "bc7be183-f52f-4562-a551-f2113d440deb", "query_weight": 1.0219354838709678 }, { "query_id": "75324c74-a8bc-43f3-8e84-bec3b36b4727", "query": "How to remove hooks added by wandb to PyTorch models?", "matching_document": "def unwatch(models=None):\n \"\"\"Remove pytorch model topology, gradient and parameter hooks.\n\n Args:\n models: (list) Optional list of pytorch models that have had watch called on them\n \"\"\"\n if models:\n if not isinstance(models, (tuple, list)):\n models = (models,)\n for model in models:\n if not hasattr(model, \"_wandb_hook_names\"):\n wandb.termwarn(\"%s model has not been watched\" % model)\n else:\n for name in model._wandb_hook_names:\n wandb.run._torch.unhook(name)\n delattr(model, \"_wandb_hook_names\")\n # TODO: we should also remove recursively model._wandb_watch_called\n\n else:\n wandb.run._torch.unhook_all()", "matching_document_document_id": "82e6e3a9-d9f0-4a47-b3ef-66fbe33dc419", "query_weight": 0.6038709677419355 }, { "query_id": "a4612c23-1928-4c19-b139-2048cc71d875", "query": "pandas DataFrame to wandb Table", "matching_document": "## Plotting with Pandas and Weights & Biases: Step-by-Step Guide\n### Integration of Weights & Biases With Pandas for Interactive and Collaborative Data Plotting\n\nIt is also extremely simple to log any Pandas dataframe to a workspace by converting it into a W&B Table: \n\n```\nimport wandb\nimport pandas as pd\n\n# Read our CSV into a new DataFrame\npandas_dataframe = pd.read_csv(\"data.csv\")\n\n# Convert the DataFrame into a W&B Table\nwandb_table = wandb.Table(dataframe=pandas_dataframe)\n\n# Add the table to an Artifact to increase the row\nwandb_table_artifact = wandb.Artifact(\n\"wandb_artifact\",\ntype=\"dataset\")\nwandb_table_artifact.add(wandb_table, \"table\")\n\n# Log the raw csv file within an artifact to preserve our data\nwandb_table_artifact.add_file(\"data.csv\")\n\n# Start a W&B run to log data\nrun = wandb.init(project=\"...\")\n\n# Log the table to visualize with a run...\nrun.log({\"data\": wandb_table})\n\n# and Log as an Artifact\nrun.log_artifact(wandb_table_artifact)\n```", "matching_document_document_id": "b584a5a0-834b-4d77-b8e3-86ce64c829d1", "query_weight": 0.8361290322580645 }, { "query_id": "fc5c8b32-c7a5-403f-a534-5c2976ea2360", "query": "How to access W&B artifacts without logging in?", "matching_document": "## Anonymous Mode\n\nAre you publishing code that you want anyone to be able to run easily? Use Anonymous Mode to let someone run your code, see a W&B dashboard, and visualize results without needing to create a W&B account first. \n\nAllow results to be logged in Anonymous Mode with `wandb.init(`**`anonymous=\"allow\"`**`)` \n\n:::info\n**Publishing a paper?** Please cite W&B, and if you have questions about how to make your code accessible while using W&B, reach out to us at support@wandb.com.\n::: \n\n### How does someone without an account see results? \n\nIf someone runs your script and you have to set `anonymous=\"allow\"`: \n\n### What happens to users with existing accounts? \n\nIf you set `anonymous=\"allow\"` in your script, we will check to make sure there's not an existing account first, before creating an anonymous account. This means that if a W&B user finds your script and runs it, their results will be logged correctly to their account, just like a normal run. \n\n## Example usage", "matching_document_document_id": "b16c8971-a81d-4473-a2b7-81b03075d7c4", "query_weight": 1.3935483870967742 }, { "query_id": "c82ef48d-f2b1-4820-85c5-dea51d75f057", "query": "run resuming Weights & Biases", "matching_document": "## Resume Runs\n#### Resume Guidance\n1. As described above, runs can be resumed by passing`resume=True` to `wandb.init()`. This can be thought of as auto-resuming, where we \u201cautomatically\u201d pick up from where an aborted run left off. If your process doesn't exit successfully, the next time you run it W&B will start logging from the last step.\n2. Note: This only works if you are running your script in the same directory as the one that failed as the file is stored at: `wandb/wandb-resume.json`.\n3. The other form of resume requires you to provide the actual run id: `wandb.init(id=run_id)` and then when you resume (if you want to be sure that it is resuming, you do `wandb.init(id=run_id, resume=\"must\")`.\n4. You can also have full control over resuming if you manage the `run_id`. We provide a utility to generate `run_id`: `wandb.util.generate_id()`. As long as you set the id to one of these unique ids for each unique run, you can say `resume=\"allow\"` and W&B will automatically resume the run with that id.More context regarding automatic and controlled resuming can be found in this section.\n2. `wandb.restore`\n\\* This will allow you to log new historical values for your metrics to a run starting from where you left off but does not take care of re-establishing the state of your code, you will need to make sure you have written checkpoints that you can load!\n\\* You can use `wandb.save` to record the state of your run via checkpoint files. Create a checkpoint file through `wandb.save()`, which can then be used through `wandb.init(resume=)`. This report illustrates how to save and restore models with W&B.", "matching_document_document_id": "3effe167-849e-453c-a198-3e826eb7e84a", "query_weight": 0.5574193548387096 }, { "query_id": "fa794c7e-df97-482d-abd3-610822e581dd", "query": "Integration of Weights & Biases with large language models for app development", "matching_document": "## Prompts for LLMs\n\nW&B Prompts is a suite of LLMOps tools built for the development of LLM-powered applications. Use W&B Prompts to visualize and inspect the execution flow of your LLMs, analyze the inputs and outputs of your LLMs, view the intermediate results and securely store and manage your prompts and LLM chain configurations. \n\n## Use Cases \n\nW&B Prompts provides several solutions for building and monitoring LLM-based apps. Software developers, prompt engineers, ML practitioners, data scientists, and other stakeholders working with LLMs need cutting-edge tools to: \n\n* Explore and debug LLM chains\u00a0and prompts with greater granularity.\n* Monitor and observe LLMs to better understand and evaluate performance, usage, and budgets. \n\n## Products \n\n### Traces \n\nW&B\u2019s LLM tool is called\u00a0*Traces*.\u00a0**Traces**\u00a0allow you to track and visualize the inputs and outputs, execution flow, model architecture, and any intermediate results of your LLM chains. \n\n### Weave \n\n### How it works \n\n## Integrations", "matching_document_document_id": "99fd9bc0-a4ff-4291-9e1c-c6aab2b9b31a", "query_weight": 0.9754838709677419 }, { "query_id": "057f4dc9-311c-4bd2-a806-f64cb98a6e66", "query": "Managing dataset versions in Weights & Biases", "matching_document": "## Dataset Versioning\n### Core Artifacts features\n1. **Upload**: Start tracking and versioning any data (files or directories) with `run.log_artifact()`. You can also track datasets in a remote filesystem (e.g. cloud storage in S3 or GCP) by reference, using a link or URI instead of the raw contents.\n2. **Version**: Define an artifact by giving it a type (`\"raw_data\"`, `\"preprocessed_data\"`, `\"balanced_data\"`) and a name (`\"imagenet_cats_10K\"`). When you log the same name again, W&B automatically creates a new version of the artifact with the latest contents.\n3. **Alias**: Set an alias like `\"best\"` or `\"production\"` to highlight the important versions in a lineage of artifacts.\n4. **Compare**: Select any two versions to browse the contents side-by-side. We're also working on a tool for dataset visualization, learn more here.\n5. **Download**: Obtain a local copy of the artifact or verify the contents by reference. \n\nFor more detail on these features, check out Artifacts how it works.", "matching_document_document_id": "7407e418-a665-438f-b605-d5dc12e4240e", "query_weight": 1.3006451612903225 }, { "query_id": "fdef0bdf-4319-41d9-94c1-930ab73d39d9", "query": "How to log prediction data to an existing W&B run after the training has finished?", "matching_document": "## An Example of Audio Classification Using Keras\n### Step 13: Saving the New Model's Predictions into The W&B Table\n\nWe will save the new model predictions after the training process. Moving on, we will log the complete table into W&B. We will use the same 5 test cases used above as our benchmark. \n\n```\n# Make predictions after training\nfor file_path, true_label in test_files:\npredicted_label_after = make_predictions(model, le, file_path)\nwandb_table.add_data(file_path, true_label, old_predictions[file_path], predicted_label_after)\n\n\n# Log the table to W&B\nwandb.log({\"Predictions\": wandb_table})\n```", "matching_document_document_id": "6d8ec863-c8fa-4ca8-b1bc-64be92d665eb", "query_weight": 1.2541935483870967 }, { "query_id": "d42335b8-9bc1-4647-b72c-ff13c1ad7e2c", "query": "How does the 'auto' setting for 'resume' in wandb.init handle run IDs and potential crashes?", "matching_document": "| `resume` | (bool, str, optional) Sets the resuming behavior. Options: `\"allow\"`, `\"must\"`, `\"never\"`, `\"auto\"` or `None`. Defaults to `None`. Cases: - `None` (default): If the new run has the same ID as a previous run, this run overwrites that data. - `\"auto\"` (or `True`): if the previous run on this machine crashed, automatically resume it. Otherwise, start a new run. - `\"allow\"`: if id is set with `init(id=\"UNIQUE_ID\")` or `WANDB_RUN_ID=\"UNIQUE_ID\"` and it is identical to a previous run, wandb will automatically resume the run with that id. Otherwise, wandb will start a new run. - `\"never\"`: if id is set with `init(id=\"UNIQUE_ID\")` or `WANDB_RUN_ID=\"UNIQUE_ID\"` and it is identical to a previous run, wandb will crash. - `\"must\"`: if id is set with `init(id=\"UNIQUE_ID\")` or `WANDB_RUN_ID=\"UNIQUE_ID\"` and it is identical to a previous run, wandb will automatically resume the run with the id. Otherwise, wandb will crash. See our guide to resuming runs for more. |\n| `reinit` | (bool, optional) Allow multiple `wandb.init()` calls in the same process. (default: `False`) |\n| `magic` | (bool, dict, or str, optional) The bool controls whether we try to auto-instrument your script, capturing basic details of your run without you having to add more wandb code. (default: `False`) You can also pass a dict, json string, or yaml filename. |\n| `config_exclude_keys` | (list, optional) string keys to exclude from `wandb.config`. |\n| `config_include_keys` | (list, optional) string keys to include in `wandb.config`. |", "matching_document_document_id": "01c9bdf0-fd26-4350-8de1-e75588077941", "query_weight": 1.0219354838709678 }, { "query_id": "4ba42cda-ff06-4766-9da3-18527018c233", "query": "What production-grade features are not available in a free personal license for W&B server?", "matching_document": "## W&B Server\n\nDeploy W&B in a resource isolated environment managed by W&B or by yourself. W&B Server is shipped as a packaged Docker image that can be deployed easily into any underlying infrastructure. There are several ways to install and host the W&B Server in different environments. \n\n:::info\nProduction-grade features for W&B Server are available for enterprise-tier only. \n\nSee the Basic Setup guide to set up a developer or trial environment.\n::: \n\nWith W&B Server you can configure and leverage features such as: \n\n* Secure Storage Connector\n* Single Sign-On\n* Role-based Access Control via LDAP\n* Audit Logs\n* User Management\n* Prometheus Monitoring\n* Slack Alerts\n* Garbage Collect Deleted Artifacts \n\nThe following sections of the documentation describes different options on how to install W&B Server, the shared responsibility model, step-by-step installation and configuration guides. \n\n## Recommendations \n\nW&B recommends the following when configuring W&B Server: \n\n## System Requirements", "matching_document_document_id": "c33e5c8b-7e45-4658-a649-ec7806312547", "query_weight": 0.7896774193548387 }, { "query_id": "beb32484-1e20-4b41-b521-c4a1046ad400", "query": "Best practices for wandb logging with multiple nodes", "matching_document": "## \u270d\ufe0f W&B Best Practices\n\n1. **Projects**: Log multiple runs to a project to compare them. `wandb.init(project=\"project-name\")`\n2. **Groups**: For multiple processes or cross validation folds, log each process as a runs and group them together. `wandb.init(group=\"experiment-1\")`\n3. **Tags**: Add tags to track your current baseline or production model.\n4. **Notes**: Type notes in the table to track the changes between runs.\n5. **Reports**: Take quick notes on progress to share with colleagues and make dashboards and snapshots of your ML projects.", "matching_document_document_id": "aa2dbfa2-471e-4ec7-b1a2-60a608aa6873", "query_weight": 0.7432258064516128 }, { "query_id": "4dd6a5d2-9cd1-4b53-b1b7-7c02822c18c9", "query": "Weights & Biases team management", "matching_document": "## Teams\n### Team Roles and Permissions\n* **Admin**: Team admins can add and remove other admins or team members. They have permissions to modify all projects and full deletion permissions. This includes, but is not limited to, deleting runs, projects, artifacts, and sweeps.\n* **Member**: A regular member of the team. A team member is invited by email by the team admin. A team member cannot invite other members. Team members can only delete runs and sweep runs created by that member. Suppose you have two members A and B. Member B moves a Run from team B's project to a different project owned by Member A. Member A can not delete the Run Member B moved to Member A's project. Only the member that creates the Run, or the team admin, can delete the run.\n* **Service**: A service worker, an API key useful for using W&B with your run automation tools. If you use the API key from a service account for your team, make sure to set the environment variable **WANDB\\_USERNAME** to attribute runs to the correct user.\n* **View-Only (Enterprise-only feature)**: View-Only members can view assets within the team such as runs, reports, and workspaces. They can follow and comment on reports, but they can not create, edit, or delete project overview, reports, or runs. View-Only members do not have an API key.", "matching_document_document_id": "a7c2233e-d024-4e8b-98ef-bc9af7bc6934", "query_weight": 0.1 }, { "query_id": "4d120cb3-7a08-446b-ba5b-68ab4efe5f6b", "query": "How do I start a project with wandb?", "matching_document": "## wandb init\n\n**Usage** \n\n`wandb init [OPTIONS]` \n\n**Summary** \n\nConfigure a directory with Weights & Biases \n\n**Options** \n\n| **Option** | **Description** |\n| --- | --- |\n| -p, --project | The project to use. |\n| -e, --entity | The entity to scope the project to. |\n| --reset | Reset settings |\n| -m, --mode | Can be \"online\", \"offline\" or \"disabled\". Defaults to online. |", "matching_document_document_id": "ab01e901-8752-4cc2-84e5-9921b115a621", "query_weight": 0.9290322580645161 }, { "query_id": "21dc836f-fffc-4e15-9a05-62b150c7423f", "query": "Is there a guide for managing team permissions on Weights & Biases?", "matching_document": "## Teams\n### Team Roles and Permissions\n* **Admin**: Team admins can add and remove other admins or team members. They have permissions to modify all projects and full deletion permissions. This includes, but is not limited to, deleting runs, projects, artifacts, and sweeps.\n* **Member**: A regular member of the team. A team member is invited by email by the team admin. A team member cannot invite other members. Team members can only delete runs and sweep runs created by that member. Suppose you have two members A and B. Member B moves a Run from team B's project to a different project owned by Member A. Member A can not delete the Run Member B moved to Member A's project. Only the member that creates the Run, or the team admin, can delete the run.\n* **Service**: A service worker, an API key useful for using W&B with your run automation tools. If you use the API key from a service account for your team, make sure to set the environment variable **WANDB\\_USERNAME** to attribute runs to the correct user.\n* **View-Only (Enterprise-only feature)**: View-Only members can view assets within the team such as runs, reports, and workspaces. They can follow and comment on reports, but they can not create, edit, or delete project overview, reports, or runs. View-Only members do not have an API key.", "matching_document_document_id": "a7c2233e-d024-4e8b-98ef-bc9af7bc6934", "query_weight": 2.136774193548387 }, { "query_id": "9df23af2-5224-4baa-8393-52e93fbdf319", "query": "Logging custom data types in Weights & Biases", "matching_document": "## Managing and Tracking ML Experiments With W&B\n### Logging Advanced\u00a0Things\u200b\n\nOne of the coolest things about W&B is that you can literally log anything. You can log custom metrics, matplotlib plots, datasets, embeddings from your models, prediction distribution, etc. \n\nRecently, Weights & Biases announced the Tables feature, which allows you to log, query and analyze tabular data. You can even visualize model predictions and compare them across models. For example: see the image below (taken from[ W&B Docs](http://docs.wandb.ai)), which compares two segmentation models. \n\nYou can log audio data, images, histograms, text, video, and tabular data and visualize/inspect them interactively. To learn more about W&B Tables, go through their [documentation](https://docs.wandb.ai/guides/data-vis). \n\nYou can even export the dashboard in CSV files to analyze them further. W&B also supports exports in PNG, SVG, PDF, and CSV, depending on the type of data you are trying to export.", "matching_document_document_id": "5b04907e-a04e-40a7-939f-b745aa031559", "query_weight": 1.8116129032258064 }, { "query_id": "536b6e8c-2cc2-4671-938e-a8c21d481f10", "query": "WandbCallback", "matching_document": "\"\"\"`WandbCallback` automatically integrates keras with wandb.\n\n Example:\n ```python\n model.fit(\n X_train,\n y_train,\n validation_data=(X_test, y_test),\n callbacks=[WandbCallback()],\n )\n ```\n\n `WandbCallback` will automatically log history data from any\n metrics collected by keras: loss and anything passed into `keras_model.compile()`.\n\n `WandbCallback` will set summary metrics for the run associated with the \"best\" training\n step, where \"best\" is defined by the `monitor` and `mode` attributes. This defaults\n to the epoch with the minimum `val_loss`. `WandbCallback` will by default save the model\n associated with the best `epoch`.\n\n `WandbCallback` can optionally log gradient and parameter histograms.\n\n `WandbCallback` can optionally save training and validation data for wandb to visualize.", "matching_document_document_id": "fc996627-d968-49a8-b58c-c4f020367bd4", "query_weight": 0.1 }, { "query_id": "cd04301e-bd22-4712-97ce-663d3096961f", "query": "Weights & Biases team features", "matching_document": "## Teams\n\nUse W&B Teams as a central workspace for your ML team to build better models faster. \n\n* **Track all the experiments** your team has tried so you never duplicate work.\n* **Save and reproduce** previously trained models.\n* **Share progress** and results with your boss and collaborators.\n* **Catch regressions** and immediately get alerted when performance drops.\n* **Benchmark model performance** and compare model versions. \n\n## Create a collaborative team \n\n1. **Sign up or log in** to your free W&B account.\n2. Click **Invite Team** in the navigation bar.\n3. Create your team and invite collaborators. \n\n:::info\n**Note**: Only the admin of an organization can create a new team.\n::: \n\n## Invite team members \n\n## Create a Team Profile \n\nYou can customize your team's profile page to show an introduction and showcase reports and projects that are visible to the public or team members. Present reports, projects, and external links. \n\n## Remove team members \n\n## Team Roles and Permissions", "matching_document_document_id": "d2f20931-cbc4-48cf-9c3e-4c52be6349b2", "query_weight": 0.1 }, { "query_id": "e40c5dd3-6596-4014-8c52-54dfc77ef3d1", "query": "Weights & Biases LLM training", "matching_document": "--- \n\n## displayed\\_sidebar: default \n\nimport { CTAButtons } from '@site/src/components/CTAButtons/CTAButtons.tsx'; \n\n# Hugging Face Transformers \n\nThe Hugging Face Transformers library makes state-of-the-art NLP models like BERT and training techniques like mixed precision and gradient checkpointing easy to use. The W&B integration adds rich, flexible experiment tracking and model versioning to interactive centralized dashboards without compromising that ease of use. \n\n## \ud83e\udd17 Next-level logging in few lines \n\n```\nos.environ[\"WANDB\\_PROJECT\"] = \"\" # name your W&B project\nos.environ[\"WANDB\\_LOG\\_MODEL\"] = \"checkpoint\" # log all model checkpoints\n\nfrom transformers import TrainingArguments, Trainer\n\nargs = TrainingArguments(..., report\\_to=\"wandb\") # turn on W&B logging\ntrainer = Trainer(..., args=args)\n\n``` \n\n:::info\nIf you'd rather dive straight into working code, check out this Google Colab.\n::: \n\n## Getting started: track experiments \n\n### 2) Name the project", "matching_document_document_id": "fe7a80f3-1c69-44cf-ab7c-fa24ac3e77dd", "query_weight": 1.207741935483871 }, { "query_id": "2cc2d4b6-184f-49b2-b435-e7706ab5ca45", "query": "Overview of wandb features", "matching_document": "## Weights & Biases overview\nW&B is a platform that helps data scientists track their models, datasets, system information and more. With a few lines of code, you can start tracking everything about these features. It's free for personal use. Team use is normally a paid utility, but teams for academic purposes are free. You can use W&B with your favourite framework, like TensorFlow, Keras, PyTorch, Sklearn, fastai and many others. \n\nAll tracking information is sent to a dedicated project page on the W&B UI, where you can open high quality visualizations, aggregate information and compare models or parameters. One of the advantages of remotely storing the experiment\u2019s information is that it is easy to collaborate on the same project and share the results with your teammates. \n\nW&B provides 4 useful tools: \n\n* Dashboard: Experiment tracking\n* Artifacts: Dataset versioning, model versioning\n* Sweeps: Hyperparameter optimization\n* Reports: Save and share reproducible findings", "matching_document_document_id": "8c1600ad-9b6e-45cd-86ae-03afd8f80332", "query_weight": 0.9290322580645161 }, { "query_id": "4e923e6f-3d28-46b1-a2af-f0cce05b15df", "query": "wandb API reference", "matching_document": "## Api\n\nView source on GitHub \n\nUsed for querying the wandb server. \n\n```\nApi(\noverrides=None,\ntimeout: Optional[int] = None,\napi\\_key: Optional[str] = None\n) -> None\n\n``` \n\n#### Examples: \n\nMost common way to initialize \n\n```\n>>> wandb.Api()\n\n``` \n\n| Arguments | |\n| --- | --- |\n| `overrides` | (dict) You can set `base_url` if you are using a wandb server other than . You can also set defaults for `entity`, `project`, and `run`. | \n\n| Attributes | |\n| --- | --- |\n| | | \n\n## Methods \n\n### `artifact` \n\nView source \n\n```\nartifact(\nname, type=None\n)\n\n``` \n\nReturn a single artifact by parsing path in the form `entity/project/name`. \n\n| Arguments | |\n| --- | --- |\n| `name` | (str) An artifact name. May be prefixed with entity/project. Valid names can be in the following forms: name:version name:alias |\n| `type` | (str, optional) The type of artifact to fetch. | \n\n| Returns | |\n| --- | --- |\n| A `Artifact` object. | | \n\n### `artifact_type` \n\nView source \n\n### `flush`", "matching_document_document_id": "8d4ccf38-8646-42b5-abfa-b46e0c4ca9f5", "query_weight": 0.1 }, { "query_id": "6a4ead63-67e3-4ca1-8aee-5258746ff8e9", "query": "Methods to reduce storage consumption by artifacts in Weights & Biases?", "matching_document": "## Storage\n\nIf you are approaching or exceeding your storage limit, there are multiple paths forward to manage your data. The path that's best for you will depend on your account type and your current project setup. \n\n## Manage storage consumption \n\nW&B offers different methods of optimizing your storage consumption: \n\n* Use\u00a0reference artifacts\u00a0to track files saved outside the W&B system, instead of uploading them to W&B storage.\n* Use an external cloud storage bucket for storage. *(Enterprise only)* \n\n## Delete data \n\nYou can also choose to delete data to remain under your storage limit. There are several ways to do this: \n\n* Delete data interactively with the app UI.\n* Set a TTL policy on Artifacts so they are automatically deleted.", "matching_document_document_id": "6dea2d7d-58f2-4830-8dc1-27b3bf719859", "query_weight": 0.1 }, { "query_id": "1d664478-1bc5-409c-9b03-eac8ebcba2f0", "query": "Are there any examples of LangChain being used with Weights & Biases?", "matching_document": "## W&B Prompts\n### Maths with LangChain\n```\nllm = ChatOpenAI(temperature=0)\ntools = load_tools([\"llm-math\"], llm=llm)\nmath_agent = initialize_agent(tools,\nllm,\nagent=AgentType.ZERO_SHOT_REACT_DESCRIPTION)\n\n``` \n\nUse LangChain as normal by calling your Agent. \n\nYou will see a Weights & Biases run start and you will be asked for your Weights & Biases API key. Once your enter your API key, the inputs and outputs of your Agent calls will start to be streamed to the Weights & Biases App. \n\n```\n# some sample maths questions\nquestions = [\n\"Find the square root of 5.4.\",\n\"What is 3 divided by 7.34 raised to the power of pi?\",\n\"What is the sin of 0.47 radians, divided by the cube root of 27?\"\n]\n\nfor question in questions:\ntry:\n# call your Agent as normal\nanswer = math_agent.run(question)\nprint(answer)\nexcept Exception as e:\n# any errors will be also logged to Weights & Biases\nprint(e)\npass\n\n``` \n\nOnce each Agent execution completes, all calls in your LangChain object will be logged to Weights & Biases \n\n### LangChain Context Manager", "matching_document_document_id": "0ec28c9a-b29d-492c-b79e-54e1f2a45b90", "query_weight": 1.2541935483870967 }, { "query_id": "7ce86a06-f18a-4f8b-88e3-149a50ce3636", "query": "How to use WandB", "matching_document": "## Quickstart\n\nInstall W&B and start tracking your machine learning experiments in minutes. \n\n## 1. Create an account and install W&B \n\nBefore you get started, make sure you create an account and install W&B: \n\n1. Sign up for a free account at and then log in to your wandb account.\n2. Install the wandb library on your machine in a Python 3 environment using `pip`. \n\nThe following code snippets demonstrate how to install and log into W&B using the W&B CLI and Python Library: \n\nInstall the CLI and Python library for interacting with the Weights and Biases API: \n\n```\npip install wandb\n\n``` \n\nInstall the CLI and Python library for interacting with the Weights and Biases API: \n\n```\n!pip install wandb\n\n``` \n\n## 2. Log in to W&B \n\nNext, log in to W&B: \n\n```\nwandb login\n\n``` \n\nOr if you are using W&B Server (including **Dedicated Cloud** or **Self-managed**): \n\n```\nwandb login --relogin --host=http://your-shared-local-host.com\n\n``` \n\n## 3. Start a run and track hyperparameters", "matching_document_document_id": "4c3d69bd-11c2-4856-87e5-bcc4fa96d3ab", "query_weight": 0.9290322580645161 }, { "query_id": "9ef16433-fb34-411a-85f3-16dc901892e9", "query": "Error communicating with wandb process", "matching_document": "## Experiments FAQ\n\nFrequently Asked Questions About Experiments \n\nThe proceeding questions are commonly asked questions about W&B Artifacts. \n\n### How do I launch multiple runs from one script? \n\nUse `wandb.init` and `run.finish()` to log multiple Runs from one script: \n\n1. `run = wandb.init(reinit=True)`: Use this setting to allow reinitializing runs\n2. `run.finish()`: Use this at the end of your run to finish logging for that run \n\n```\nimport wandb\n\nfor x in range(10):\nrun = wandb.init(reinit=True)\nfor y in range(100):\nwandb.log({\"metric\": x + y})\nrun.finish()\n\n``` \n\nAlternatively you can use a python context manager which will automatically finish logging: \n\n```\nimport wandb\n\nfor x in range(10):\nrun = wandb.init(reinit=True)\nwith run:\nfor y in range(100):\nrun.log({\"metric\": x + y})\n\n``` \n\n### `InitStartError: Error communicating with wandb process` \n\nThis error indicates that the library is having difficulty launching the process which synchronizes data to the server.", "matching_document_document_id": "9b1af9fa-cbac-449f-a522-8d9199665bcf", "query_weight": 1.7651612903225806 }, { "query_id": "969f96f3-5db7-4687-8346-fc94c5ffc184", "query": "Transferring wandb runs from team to personal project", "matching_document": "--- \n\n## displayed\\_sidebar: default \n\n# Manage Runs \n\n### Move runs to a team \n\nOn the project page: \n\n1. Click the table tab to expand the runs table\n2. Click the checkbox to select all runs\n3. Click **Move**: the destination project can be in your personal account or any team that you're a member of. \n\n### Send new runs to a team \n\nIn your script, set the entity to your team. \"Entity\" just means your username or team name. Create an entity (personal account or team account) in the web app before sending runs there. \n\n```\nwandb.init(entity=\"example-team\")\n\n``` \n\nYour **default entity** is updated when you join a team. This means that on your settings page, you'll see that the default location to create a new project is now the team you've just joined. Here's an example of what that settings page section looks like:", "matching_document_document_id": "3e05fcaf-8f88-491a-99cd-2e550d1c9085", "query_weight": 2.136774193548387 }, { "query_id": "ebafeecb-d0c1-4525-ac51-e79c5aa99ff4", "query": "W&B run tables to Pandas DataFrame", "matching_document": "def _get_updated_result_type(self, row):\n \"\"\"Returns the updated result type based on the inputted row.\n\n Raises:\n TypeError: if the assignment is invalid.\n \"\"\"\n incoming_row_dict = {\n col_key: row[ndx] for ndx, col_key in enumerate(self.columns)\n }\n current_type = self._column_types\n result_type = current_type.assign(incoming_row_dict)\n if isinstance(result_type, _dtypes.InvalidType):\n raise TypeError(\n \"Data row contained incompatible types:\\n{}\".format(\n current_type.explain(incoming_row_dict)\n )\n )\n return result_type\n\n def get_dataframe(self):\n \"\"\"Returns a `pandas.DataFrame` of the table.\"\"\"\n pd = util.get_module(\n \"pandas\",\n required=\"Converting to pandas.DataFrame requires installing pandas\",\n )\n return pd.DataFrame.from_records(self.data, columns=self.columns)", "matching_document_document_id": "55d4d5f6-04e4-4090-bb7c-fb297871d8bc", "query_weight": 1.2541935483870967 }, { "query_id": "f35e52f9-89db-4b81-907c-f0cd815dd43e", "query": "W&B query runs", "matching_document": "## Import & Export Data\n### Export Data\n#### Querying Multiple Runs\nThe W&B API also provides a way for you to query across runs in a project with api.runs(). The most common use case is exporting runs data for custom analysis. The query interface is the same as the one MongoDB uses. \n\n```\nruns = api.runs(\n\"username/project\",\n{\"$or\": [{\"config.experiment\\_name\": \"foo\"}, {\"config.experiment\\_name\": \"bar\"}]},\n)\nprint(f\"Found {len(runs)} runs\")\n\n``` \n\nCalling `api.runs` returns a `Runs` object that is iterable and acts like a list. By default the object loads 50 runs at a time in sequence as required, but you can change the number loaded per page with the `per_page` keyword argument. \n\n`api.runs` also accepts an `order` keyword argument. The default order is `-created_at`, specify `+created_at` to get results in ascending order. You can also sort by config or summary values e.g. `summary.val_acc` or `config.experiment_name`", "matching_document_document_id": "5d30d566-2bbd-401c-9d26-b2bc5000c582", "query_weight": 0.1 }, { "query_id": "29384fdc-a931-46bc-b5cb-0b4be4245352", "query": "W&B visualization tools", "matching_document": "*W&B Datasets & Predictions is currently in the early-access phase. You can use it in our production service at wandb.ai, with some limitations. APIs are subject to change. We'd love to hear questions, comments, and ideas! Drop us a line at feedback@wandb.com.* \n\n# W&B Dataset Visualization Demo \n\nThis notebook demonstrates W&B's dataset visualization features. In particular we show how W&B Artifacts help visualize datasets and predictions, focusing on image data. We track model and data lineage and perform interactive analysis on the resulting datasets. The overall flow will be: \n\n1. Create a dataset\n2. Split the dataset into \"train\" and \"test\"\n3. Train a model on the \"train\" dataset\n4. Load the trained model and log predictions on both datasets\n5. Analyze the results in W&B's UI \n\nYou can see live interactive examples logged to W&B in this public project: Dsviz Demo. \n\n### Explore your training data \n\n### Visualize your model predictions \n\n# Step 0: Setup \n\n## Install requirements & utils", "matching_document_document_id": "8fb7c9f2-454d-4555-902e-aa52e7475599", "query_weight": 1.3470967741935482 }, { "query_id": "9485ee11-c52f-4c46-8fe7-ad16003176e5", "query": "How to correctly log images using WandbLogger?", "matching_document": "## Log Media & Objects\n### Images\n```\nimages = wandb.Image(image\\_array, caption=\"Top: Output, Bottom: Input\")\n\nwandb.log({\"examples\": images})\n\n``` \n\nWe assume the image is gray scale if the last dimension is 1, RGB if it's 3, and RGBA if it's 4. If the array contains floats, we convert them to integers between `0` and `255`. If you want to normalize your images differently, you can specify the `mode` manually or just supply a `PIL.Image`, as described in the \"Logging PIL Images\" tab of this panel. \n\nFor full control over the conversion of arrays to images, construct the `PIL.Image` yourself and provide it directly. \n\n```\nimages = [PIL.Image.fromarray(image) for image in image\\_array]\n\nwandb.log({\"examples\": [wandb.Image(image) for image in images]})\n\n``` \n\nFor even more control, create images however you like, save them to disk, and provide a filepath. \n\n```\nim = PIL.fromarray(...)\nrgb\\_im = im.convert(\"RGB\")\nrgb\\_im.save(\"myimage.jpg\")\n\nwandb.log({\"example\": wandb.Image(\"myimage.jpg\")})\n\n```", "matching_document_document_id": "cefd3fca-d245-469d-95bf-eb8b4b7f28f9", "query_weight": 0.9754838709677419 }, { "query_id": "a7e0c296-f3e4-4ecd-a838-d07ceb18f2f2", "query": "Weights & Biases sweep setup", "matching_document": "## Sweep configuration structure\n### Basic structure\ntrain.py\n```\nsweep\\_configuration = {\n\"name\": \"sweepdemo\",\n\"method\": \"bayes\",\n\"metric\": {\"goal\": \"minimize\", \"name\": \"validation\\_loss\"},\n\"parameters\": {\n\"learning\\_rate\": {\"min\": 0.0001, \"max\": 0.1},\n\"batch\\_size\": {\"values\": [16, 32, 64]},\n\"epochs\": {\"values\": [5, 10, 15]},\n\"optimizer\": {\"values\": [\"adam\", \"sgd\"]},\n},\n}\n\n``` \n\nDefine a sweep configuration in a YAML file if you want to manage sweeps interactively from the command line (CLI) \n\nconfig.yaml\n```\nprogram: train.py\nname: sweepdemo\nmethod: bayes\nmetric:\ngoal: minimize\nname: validation\\_loss\nparameters:\nlearning\\_rate:\nmin: 0.0001\nmax: 0.1\nbatch\\_size:\nvalues: [16, 32, 64]\nepochs:\nvalues: [5, 10, 15]\noptimizer:\nvalues: [\"adam\", \"sgd\"]\n\n```", "matching_document_document_id": "2a1ce862-7280-49c1-80c4-e4dfdd16914e", "query_weight": 0.5574193548387096 }, { "query_id": "e0107b7a-d97d-4d37-a102-ac5868ad98ed", "query": "when inserting wandb config and log code into python training script, at what point in the training code should i place the wandb.finish() code?", "matching_document": "## init\nIn an ML training pipeline, you could add `wandb.init()`\nto the beginning of your training script as well as your evaluation\nscript, and each piece would be tracked as a run in W&B. \n\n`wandb.init()` spawns a new background process to log data to a run, and it\nalso syncs data to wandb.ai by default, so you can see live visualizations. \n\nCall `wandb.init()` to start a run before logging data with `wandb.log()`: \n\n```\nimport wandb\n\nwandb.init()\n# ... calculate metrics, generate media\nwandb.log({\"accuracy\": 0.9})\n\n``` \n\n`wandb.init()` returns a run object, and you can also access the run object\nvia `wandb.run`: \n\n```\nimport wandb\n\nrun = wandb.init()\n\nassert run is wandb.run\n\n``` \n\nAt the end of your script, we will automatically call `wandb.finish` to\nfinalize and cleanup the run. However, if you call `wandb.init` from a\nchild process, you must explicitly call `wandb.finish` at the end of the\nchild process. \n\nFor more on using `wandb.init()`, including detailed examples, check out our\nguide and FAQs.", "matching_document_document_id": "f8079de6-06d4-4899-8cca-4333a040fcac", "query_weight": 0.6967741935483871 }, { "query_id": "19afd6fc-cbfc-4408-a58e-49a65e80b30d", "query": "Are there common mistakes to avoid when integrating wandb.finish() in a training script?", "matching_document": "## Train model with DDP\nThe preceding image demonstrates the W&B App UI dashboard. On the sidebar we see two experiments. One labeled 'null' and a second (bound by a yellow box) called 'DPP'. If you expand the group (select the Group dropdown) you will see the W&B Runs that are associated to that experiment. \n\n### Use W&B Service to avoid common distributed training issues. \n\nThere are two common issues you might encounter when using W&B and distributed training: \n\n1. **Hanging at the beginning of training** - A `wandb` process can hang if the `wandb` multiprocessing interferes with the multiprocessing from distributed training.\n2. **Hanging at the end of training** - A training job might hang if the `wandb` process does not know when it needs to exit. Call the `wandb.finish()` API at the end of your Python script to tell W&B that the Run finished. The wandb.finish() API will finish uploading data and will cause W&B to exit. \n\n### Enable W&B Service \n\n### Example use cases for multiprocessing", "matching_document_document_id": "39684e44-a845-4e76-bafe-10b5fda1cf4b", "query_weight": 0.6967741935483871 }, { "query_id": "53c7f76c-d293-436f-8a3c-b62cf0d0881f", "query": "Integrating NLP models with Weights & Biases for automatic logging", "matching_document": "## Optimize \ud83e\udd17 Hugging Face models with Weights & Biases\n\nHugging Face provides tools to quickly train neural networks for NLP (Natural Language Processing) on any task (classification, translation, question answering, etc) and any dataset with PyTorch and TensorFlow 2.0. \n\nCoupled with Weights & Biases integration, you can quickly train and monitor models for full traceability and reproducibility without any extra line of code! You just need to install the library, sign in, and your experiments will automatically be logged: \n\n```\npip install wandb\nwandb login\n\n``` \n\n**Note**: To enable logging to W&B, set `report_to` to `wandb` in your `TrainingArguments` or script. \n\nW&B integration with \ud83e\udd17 Hugging Face can automatically:\n\\* log your configuration parameters\n\\* log your losses and metrics\n\\* log gradients and parameter distributions\n\\* log your model\n\\* keep track of your code\n\\* log your system metrics (GPU, CPU, memory, temperature, etc) \n\n## Here's what the W&B interactive dashboard will look like:", "matching_document_document_id": "b42d7afe-6db5-4ce4-a388-988c7feb33e8", "query_weight": 0.9754838709677419 }, { "query_id": "31494d31-6c9a-43b9-9dd0-e1a81d2b886b", "query": "How to use artifact.add_file in Weights & Biases", "matching_document": "## ResNet Strikes Back: A Training Procedure in TIMM\n### Saving and using model artifacts with W&B for training\n> W&B Artifacts was designed to make it effortless to version your datasets and models, regardless of whether you want to store your files with W&B or whether you already have a bucket you want W&B to track. Once you've tracked your datasets or model files, W&B will automatically log each and every modification, giving you a complete and auditable history of changes to your files.\nThis lets you focus on the fun and important parts of evolving your datasets and training your models, while W&B handles the otherwise tedious process of tracking all the details. \n\nThis is another added benefit we get integrating with Weights and Biases. During training, all model files get automatically stored to W&B for future use if needed: \n\nLogging artifacts to Weights and Biases is painless! All it takes is 3 lines of code: \n\n```\nartifact = wandb.Artifact(\ufeff\ufeff, type\ufeff=\ufeff'model'\ufeff)\nartifact.add_file(\ufeff\ufeff)\nwandb.run.log_artifact(artifact)\n```", "matching_document_document_id": "492822bc-9af8-485a-ba21-c0e1c66a54be", "query_weight": 0.1 }, { "query_id": "62f4e886-5c03-452a-8a89-6b5e108608a2", "query": "WandB log method", "matching_document": "## Run\n### Methods\n#### `log`\n\nView source \n\n```\nlog(\ndata: Dict[str, Any],\nstep: Optional[int] = None,\ncommit: Optional[bool] = None,\nsync: Optional[bool] = None\n) -> None\n\n``` \n\nLog a dictionary of data to the current run's history. \n\nUse `wandb.log` to log data from runs, such as scalars, images, video,\nhistograms, plots, and tables. \n\nSee our guides to logging for\nlive examples, code snippets, best practices, and more. \n\nThe most basic usage is `wandb.log({\"train-loss\": 0.5, \"accuracy\": 0.9})`.\nThis will save the loss and accuracy to the run's history and update\nthe summary values for these metrics. \n\nVisualize logged data in the workspace at wandb.ai,\nor locally on a self-hosted instance\nof the W&B app, or export data to visualize and explore locally, e.g. in\nJupyter notebooks, with our API. \n\nIn the UI, summary values show up in the run table to compare single values across runs.\nSummary values can also be set directly with `wandb.run.summary[\"key\"] = value`.", "matching_document_document_id": "fc908478-e886-40b6-80a3-5fe44d8b4097", "query_weight": 0.1 }, { "query_id": "3b851205-3882-4e4c-8f42-073286cfe25f", "query": "Weights & Biases PyTorch example", "matching_document": "## \ud83d\udd25 = W&B \u2795 PyTorch\n\nUse Weights & Biases for machine learning experiment tracking, dataset versioning, and project collaboration. \n\n## What this notebook covers: \n\nWe show you how to integrate Weights & Biases with your PyTorch code to add experiment tracking to your pipeline. \n\n## The resulting interactive W&B dashboard will look like: \n\n## In pseudocode, what we'll do is: \n\n```\n# import the library\nimport wandb\n\n# start a new experiment\nwandb.init(project=\"new-sota-model\")\n\n#\u2003capture a dictionary of hyperparameters with config\nwandb.config = {\"learning\\_rate\": 0.001, \"epochs\": 100, \"batch\\_size\": 128}\n\n# set up model and data\nmodel, dataloader = get\\_model(), get\\_data()\n\n# optional: track gradients\nwandb.watch(model)\n\nfor batch in dataloader:\nmetrics = model.training\\_step()\n#\u2003log metrics inside your training loop to visualize model performance\nwandb.log(metrics)\n\n# optional: save model at the end\nmodel.to\\_onnx()\nwandb.save(\"model.onnx\")\n\n``` \n\n## Follow along with a video tutorial!", "matching_document_document_id": "ee62983f-f238-4b4a-bde8-6b4c068d7918", "query_weight": 1.2541935483870967 }, { "query_id": "a4aec51f-3cca-45ea-9bb0-f0c1be80ff2f", "query": "Parallel execution of Weights & Biases sweeps on multiple GPUs", "matching_document": "## Parallelize agents\n#### Parallelize on a multi-GPU machine\n1. Open more than one terminal window on your local machine.\n2. Specify the GPU instance to use with `CUDA_VISIBLE_DEVICES` when you start a W&B Sweep job (`wandb agent`). Assign `CUDA_VISIBLE_DEVICES` an integer value corresponding to the GPU instance to use. \n\nFor example, suppose you have two NVIDIA GPUs on your local machine. Open a terminal window and set `CUDA_VISIBLE_DEVICES` to `0` (`CUDA_VISIBLE_DEVICES=0`). Replace `sweep_ID` in the proceeding example with the W&B Sweep ID that is returned when you initialized a W&B Sweep: \n\nTerminal 1 \n\n```\nCUDA\\_VISIBLE\\_DEVICES=0 wandb agent sweep_ID\n\n``` \n\nOpen a second terminal window. Set `CUDA_VISIBLE_DEVICES` to `1` (`CUDA_VISIBLE_DEVICES=1`). Paste the same W&B Sweep ID for the `sweep_ID` mentioned in the proceeding code snippet: \n\nTerminal 2 \n\n```\nCUDA\\_VISIBLE\\_DEVICES=1 wandb agent sweep_ID\n\n```", "matching_document_document_id": "03228a82-a0d9-405b-ba85-e645187827e1", "query_weight": 1.207741935483871 }, { "query_id": "a68cee7b-ce44-4959-bb86-ee3da29543ff", "query": "What is the purpose of artifact.add_file in Weights & Biases?", "matching_document": "## Construct artifacts\n\nConstruct Artifacts \n\nUse the W&B Python SDK to construct artifacts from W&B Runs. You can add files, directories, URIs, and files from parallel runs to artifacts. After you add a file to an artifact, save the artifact to the W&B Server or your own private server. \n\nFor information on how to track external files, such as files stored in Amazon S3, see the Track external files page. \n\n## How to construct an artifact \n\nConstruct a W&B Artifact in three steps: \n\n### 1. Create an artifact Python object with `wandb.Artifact()` \n\nInitialize the `wandb.Artifact()` class to create an artifact object. Specify the following parameters: \n\n### 2. Add one more files to the artifact \n\nAdd files, directories, external URI references (such as Amazon S3) and more with artifact methods. For example, to add a single text file, use the `add_file` method: \n\n```\nartifact.add\\_file(local\\_path=\"hello\\_world.txt\", name=\"optional-name\")\n\n``` \n\n### 3. Save your artifact to the W&B server", "matching_document_document_id": "03e958e1-2c55-4687-8dd8-79a37aae8abf", "query_weight": 0.1 }, { "query_id": "c452725c-1312-4038-9c0d-e2cc48ca2a0d", "query": "Best practices for versioning datasets in Weights & Biases.", "matching_document": "## Dataset Versioning\n### Core Artifacts features\n1. **Upload**: Start tracking and versioning any data (files or directories) with `run.log_artifact()`. You can also track datasets in a remote filesystem (e.g. cloud storage in S3 or GCP) by reference, using a link or URI instead of the raw contents.\n2. **Version**: Define an artifact by giving it a type (`\"raw_data\"`, `\"preprocessed_data\"`, `\"balanced_data\"`) and a name (`\"imagenet_cats_10K\"`). When you log the same name again, W&B automatically creates a new version of the artifact with the latest contents.\n3. **Alias**: Set an alias like `\"best\"` or `\"production\"` to highlight the important versions in a lineage of artifacts.\n4. **Compare**: Select any two versions to browse the contents side-by-side. We're also working on a tool for dataset visualization, learn more here.\n5. **Download**: Obtain a local copy of the artifact or verify the contents by reference. \n\nFor more detail on these features, check out Artifacts how it works.", "matching_document_document_id": "7407e418-a665-438f-b605-d5dc12e4240e", "query_weight": 1.3006451612903225 }, { "query_id": "0aa7f45f-c262-4263-a0d0-3cdd49d257df", "query": "accelerate launch args", "matching_document": "## Launching Distributed Code\nHuggingFace Accelerate has a special CLI command to help you launch your code in your system through accelerate launch. This command wraps around all of the different commands needed to launch your script on various platforms without you having to remember what each of them is. \n\nYou can launch your script quickly by using: \n\n```\naccelerate launch {script_name.py} --arg1 --arg2 ...\n``` \n\nJust put accelerate launch at the start of your command, and pass in additional arguments and parameters to your script afterward like normal! \n\nSince this runs the various torch spawn methods, all of the expected environment variables can be modified here as well. For example, here is how to use accelerate launch with a single GPU: \n\n```\nCUDA_VISIBLE_DEVICES=\"0\" accelerate launch {script_name.py} --arg1 --arg2 ...\n``` \n\nTo explore more options, you can take a look at the documentation [here](https://huggingface.co/docs/accelerate/v0.13.2/en/basic_tutorials/launch).", "matching_document_document_id": "98ab6d0b-2a8a-42d4-9393-4fe0cec531d9", "query_weight": 0.6503225806451612 }, { "query_id": "8066302c-a3cf-4b1e-b83f-bd9e35e3af79", "query": "Best practices for managing artifact versions in Weights & Biases", "matching_document": "## Dataset Versioning\n### Core Artifacts features\n1. **Upload**: Start tracking and versioning any data (files or directories) with `run.log_artifact()`. You can also track datasets in a remote filesystem (e.g. cloud storage in S3 or GCP) by reference, using a link or URI instead of the raw contents.\n2. **Version**: Define an artifact by giving it a type (`\"raw_data\"`, `\"preprocessed_data\"`, `\"balanced_data\"`) and a name (`\"imagenet_cats_10K\"`). When you log the same name again, W&B automatically creates a new version of the artifact with the latest contents.\n3. **Alias**: Set an alias like `\"best\"` or `\"production\"` to highlight the important versions in a lineage of artifacts.\n4. **Compare**: Select any two versions to browse the contents side-by-side. We're also working on a tool for dataset visualization, learn more here.\n5. **Download**: Obtain a local copy of the artifact or verify the contents by reference. \n\nFor more detail on these features, check out Artifacts how it works.", "matching_document_document_id": "7407e418-a665-438f-b605-d5dc12e4240e", "query_weight": 0.1 }, { "query_id": "30d66ad6-76c1-426f-8e1f-fd60f7787a22", "query": "How to authenticate a W&B session in Python?", "matching_document": "## \ud83d\ude80 Setup\n\nStart out by installing the experiment tracking library and setting up your free W&B account: \n\n1. Install with `!pip install`\n2. `import` the library into Python\n3. `.login()` so you can log metrics to your projects \n\nIf you've never used Weights & Biases before,\nthe call to `login` will give you a link to sign up for an account.\nW&B is free to use for personal and academic projects! \n\n```\n!pip install wandb -Uq\n\n``` \n\n```\nimport wandb\n\n``` \n\n```\nwandb.login()\n\n```", "matching_document_document_id": "ee82f3a4-c71a-434e-a6fe-0170efcb2674", "query_weight": 1.2541935483870967 }, { "query_id": "28fc4f23-2a04-4237-8e70-3d936a303515", "query": "Steps to create a new dataset version in W&B.", "matching_document": "## Create new artifact versions\n### Create new artifact versions from scratch\n* **Single run**: A single run provides all the data for a new version. This is the most common case and is best suited when the run fully recreates the needed data. For example: outputting saved models or model predictions in a table for analysis.\n* **Distributed runs**: A set of runs collectively provides all the data for a new version. This is best suited for distributed jobs which have multiple runs generating data, often in parallel. For example: evaluating a model in a distributed manner, and outputting the predictions. \n\nW&B will create a new artifact and assign it a `v0` alias if you pass a name to the `wandb.Artifact` API that does not exist in your project. W&B checksums the contents when you log again to the same artifact. If the artifact changed, W&B saves a new version `v1`. \n\nW&B will retrieve an existing artifact if you pass a name and artifact type to the `wandb.Artifact` API that matches an existing artifact in your project. The retrieved artifact will have a version greater than 1.", "matching_document_document_id": "2e896de8-7ee0-41ba-983c-2ca7b0b7cb6e", "query_weight": 0.1 }, { "query_id": "58ed7656-5796-46c1-abac-d36691bea061", "query": "How to efficiently log artifacts in Weights & Biases?", "matching_document": "## Project Setup\n### Logging and Tracking\n* Create an artifact [ `artifact = wandb.Artifact('name', type='dataset/model/result')` ]\n* Add files or directories [ `artifact.add_file('model-name.pth')` ]\n* Use the artifact after logging it to the W&B dashboard if it does not already exists [`run.use_artifact(artifact)`]. This Creates an incoming directed graph.\nYou can also log the outputs( like checkpoints) to W&B dashboard[ `run.log_artifact(artifact)`]. This creates and outgoing directed graph \n\nAll the artifacts are versioned by default so that any changes made later on can easily be tracked.\nLater, we will see how these artifacts can be retrieved from the dashboard to the local machine.", "matching_document_document_id": "a2015038-4a7e-4b3c-b3a3-e954cc69b5c3", "query_weight": 1.1148387096774193 }, { "query_id": "1db40c0d-f0c2-4720-879c-121544ba7339", "query": "kfold training", "matching_document": "## What Is K-Fold Cross-Validation?\nK-fold cross-validation is a procedure where a dataset is divided into multiple training and validation sets (folds), where k is the number of them, to help safeguard the model against random bias caused by the selection of only one training and validation set. \n\nWith this procedure, we run our modeling process on different subsets of the data to get multiple measures of model quality. \n\nFor example, we could begin by dividing the data into 5 pieces, each 20% of the full dataset. In this case, we say that we have broken the data into 5 \"folds.\" \n\nThen, we run one experiment for each fold:", "matching_document_document_id": "ca05797c-e1ad-4ec0-9faa-74c4653739ad", "query_weight": 1.3470967741935482 }, { "query_id": "aee4b4f6-71db-4d16-9244-b723877b2dfa", "query": "What formats can chart data be exported as using the Weights & Biases API?", "matching_document": "\" \n\n# Export Your Data from W&B \n\nDescription: Learn how to export CSV, PDF, and LaTeX from W&B \n\nBody: \n\nWe're excited to announce our latest highly requested feature! You can now export tables, graphs and reports to CSV, LaTeX and PDF. Here are some examples of how you can export data easily from the Weights & Biases UI. \n\nIn any graph, click on the arrow in the upper right and click \u201cPanel Export\u201d. \n\nYou\u2019ll see a popup table with the underlying data - click Save as CSV to start the CSV download. \n\nIn a table click the \u201cExport Table\u201d button in the upper right corner. \n\nIn a report, click the \u201cDownload as LaTeX\u201d link. \n\nThis will download a zip file with a report.tex and images of your panels attached. The .tex file is a latex file that you can modify to style your document. You can load the zip file into Overleaf.com and use their visual editor, or you can download pdflatex and run \u2018pdflatext report.tex\u2019 to generate a report.pdf file. \n\nHere\u2019s an example of an exported LaTeX report: \n\n\"", "matching_document_document_id": "78c84355-d5cd-4b5f-920d-509ab7bf3ad9", "query_weight": 1.8116129032258064 }, { "query_id": "09d53006-12ed-4cce-8d42-aee2fbf0d260", "query": "multi-GPU sweep setup", "matching_document": "## Multi-GPU Hyperparameter Sweeps in Three Simple Steps\nSetting up the infrastructure for these sweeps can be tedious. So we've built W&B sweeps to be simple to set up and flexible to deploy. Inspired by [Google's Vizier](https://web.archive.org/web/20210501083721/https://ai.google/research/pubs/pub46180), we've implemented a wide range of features, including bayesian optimization and [hyperband](https://web.archive.org/web/20210501083721/https://arxiv.org/abs/1603.06560) early stopping. Integration is simple: if you have a machine learning script running on the command line, you\u2019re ready to go. \n\n### Step 1: Select Hyperparameters \n\nFirst, you\u2019ll want to select the hyperparameters you\u2019re sweeping over. Set this up in a YAML file, as detailed further in the sweep docs. \n\n```\nwandb init # Initialize your project repo\nwandb sweep sweep.yaml # returns your SWEEP_ID\n``` \n\n### Step 2: Launch Agents \n\nGrab your sweep ID from the output of the command above and launch some agents to begin running your sweep. \n\n```\nwandb agent mcg70107\n```", "matching_document_document_id": "32fdc30f-fe0c-4b05-8419-67aa9837edb9", "query_weight": 1.207741935483871 }, { "query_id": "e0c7897e-b05c-413c-8e92-133380c39e92", "query": "Weights & Biases integration with LangChain documentation", "matching_document": "## Prompts Quickstart\n### Use W&B Trace with LangChain\n#### 1. Set the LANGCHAIN\\_WANDB\\_TRACING environment variable\nFirst, set the LANGCHAIN\\_WANDB\\_TRACING environment variable to true. This will turn on automated Weights & Biases logging with LangChain: \n\n```\nimport os\n\n# turn on wandb logging for langchain\nos.environ[\"LANGCHAIN\\_WANDB\\_TRACING\"] = \"true\"\n\n``` \n\nThats it! Now any call to a LangChain LLM, Chain, Tool or Agent will be logged to Weights & Biases.", "matching_document_document_id": "6adad621-230b-42f0-badc-1738fef555a6", "query_weight": 1.2541935483870967 }, { "query_id": "52184b94-f6b4-4795-ab6d-6e64be9c9386", "query": "Integration of Weights & Biases with LLMs for app development", "matching_document": "**Weights & Biases Prompts** is a suite of LLMOps tools built for the development of LLM-powered applications. \n\nUse W&B Prompts to visualize and inspect the execution flow of your LLMs, analyze the inputs and outputs of your LLMs, view the intermediate results and securely store and manage your prompts and LLM chain configurations. \n\n#### \ud83e\ude84 View Prompts In Action \n\n**In this notebook we will demostrate W&B Prompts:** \n\n* Using our 1-line LangChain integration\n* Using our Trace class when building your own LLM Pipelines \n\nSee here for the full W&B Prompts documentation \n\n## Installation \n\n```\n!pip install \"wandb>=0.15.4\" -qqq\n!pip install \"langchain>=0.0.218\" openai -qqq\n\n``` \n\n```\nimport langchain\nassert langchain.__version__ >= \"0.0.218\", \"Please ensure you are using LangChain v0.0.188 or higher\"\n\n``` \n\n## Setup \n\nThis demo requires that you have an OpenAI key \n\n# W&B Prompts \n\nW&B Prompts consists of three main components: \n\n**Trace table**: Overview of the inputs and outputs of a chain.", "matching_document_document_id": "5ccb89d6-6353-4b96-9920-060f32aad458", "query_weight": 1.207741935483871 }, { "query_id": "cb9da279-fc89-4669-9042-8b610b0722d0", "query": "How to log model name in Weights & Biases?", "matching_document": "## Log Models\n### Log a model to a W&B run\nUse the `log_model` to log a model artifact that contains content within a directory you specify. The `log_model` method also marks the resulting model artifact as an output of the W&B run. \n\nYou can track a model's dependencies and the model's associations if you mark the model as the input or output of a W&B run. View the lineage of the model within the W&B App UI. See the Explore and traverse artifact graphs page within the Artifacts chapter for more information. \n\nProvide the path where your model file(s) are saved to the `path` parameter. The path can be a local file, directory, or reference URI to an external bucket such as `s3://bucket/path`. \n\nEnsure to replace values enclosed in `<>` with your own. \n\n```\nimport wandb\n\n# Initialize a W&B run\nrun = wandb.init(project=\"\", entity=\"\")\n\n# Log the model\nrun.log\\_model(path=\"\", name=\"\")\n\n```", "matching_document_document_id": "7516c154-8b82-416e-97d1-eeb5994a0ed9", "query_weight": 1.8116129032258064 }, { "query_id": "630ddc22-1ae7-4ba9-b9a8-706914fdcbb5", "query": "wandb version compatibility", "matching_document": "def check_wandb_version(api: Api) -> None:\n print(\"Checking wandb package version is up to date\".ljust(72, \".\"), end=\"\")\n _, server_info = api.viewer_server_info()\n fail_string = None\n warning = False\n max_cli_version = server_info.get(\"cliVersionInfo\", {}).get(\"max_cli_version\", None)\n min_cli_version = server_info.get(\"cliVersionInfo\", {}).get(\n \"min_cli_version\", \"0.0.1\"\n )\n if parse_version(wandb.__version__) < parse_version(min_cli_version):\n fail_string = \"wandb version out of date, please run pip install --upgrade wandb=={}\".format(\n max_cli_version\n )\n elif parse_version(wandb.__version__) > parse_version(max_cli_version):\n fail_string = (\n \"wandb version is not supported by your local installation. This could \"\n \"cause some issues. If you're having problems try: please run `pip \"\n f\"install --upgrade wandb=={max_cli_version}`\"\n )\n warning = True\n\n print_results(fail_string, warning)", "matching_document_document_id": "13d4de2e-232b-485c-b565-2e6a652f600e", "query_weight": 1.1148387096774193 }, { "query_id": "2fedb0d9-3e58-4d5b-8392-6397ac8ec28b", "query": "GradientTape TensorFlow", "matching_document": "## Customizing Training Loops in TensorFlow 2.0\n### What is a customized training loop?\n* `tf.GradientTape` context lets you watch the trainable variables of your model and records the operations for performing automatic differentiation. More information .\n* Next two steps are normal - you run your input features through your model and calculate the loss.\n* Now, `gradients = tape.gradient(loss, model.trainable_variables)` actually does the differentiation and gets you the gradients of the loss function with respect to the parameters i.e. trainable variables of your model. Note that the necessary inter-dependency (for differentiation) between the loss function and trainable variables is automatically managed by the `tf.GradientTape` context.\n* `optimizer.apply_gradients(zip(gradients, model.trainable_variables))` performs the parameter updates in the model. \n\nAnd that\u2019s it! \n\nThis is a rough simulation of the classic fit function provided by Keras but notice that we now have the flexibility to control how we want the parameter updates to take place in our model among many other things.", "matching_document_document_id": "7237e928-72f6-4528-bca0-a94cfc8ffb97", "query_weight": 1.161290322580645 }, { "query_id": "4b876371-d460-47dd-af8d-89c36aa5e9b4", "query": "Manage storage settings for W&B artifacts", "matching_document": "## Storage\n\nArtifact Storage \n\nW&B stores artifact files in a private Google Cloud Storage bucket located in the United States by default. All files are encrypted at rest and in transit. \n\nFor sensitive files, we recommend you set up Private Hosting or use reference artifacts. \n\nDuring training, W&B locally saves logs, artifacts, and configuration files in the following local directories: \n\n| File | Default location | To change default location set: |\n| --- | --- | --- |\n| logs | `./wandb` | `dir` in `wandb.init` or set the `WANDB_DIR` environment variable |\n| artifacts | `~/.cache/wandb` | the `WANDB_CACHE_DIR` environment variable |\n| configs | `~/.config/wandb` | the `WANDB_CONFIG_DIR` environment variable | \n\n:::caution\nDepending on the machine on `wandb` is initialized on, these default folders may not be located in a writeable part of the file system. This might trigger an error.\n::: \n\n### Clean up local artifact cache \n\n### How much storage does each artifact version use?", "matching_document_document_id": "2ec0a152-2b53-4183-8398-b61730217aac", "query_weight": 1.3935483870967742 }, { "query_id": "e03024f1-89b5-4e5b-9ce3-08c0e8d6f1e0", "query": "WandB sweep API documentation", "matching_document": "## Sweep\n\nView source on GitHub \n\nA set of runs associated with a sweep. \n\n```\nSweep(\nclient, entity, project, sweep\\_id, attrs=None\n)\n\n``` \n\n#### Examples: \n\nInstantiate with: \n\n```\napi = wandb.Api()\nsweep = api.sweep(path/to/sweep)\n\n``` \n\n| Attributes | |\n| --- | --- |\n| `runs` | (`Runs`) list of runs |\n| `id` | (str) sweep id |\n| `project` | (str) name of project |\n| `config` | (str) dictionary of sweep configuration |\n| `state` | (str) the state of the sweep |\n| `expected_run_count` | (int) number of expected runs for the sweep | \n\n## Methods \n\n### `best_run` \n\nView source \n\n```\nbest\\_run(\norder=None\n)\n\n``` \n\nReturn the best run sorted by the metric defined in config or the order passed in. \n\n### `display` \n\nView source \n\n```\ndisplay(\nheight=420, hidden=(False)\n) -> bool\n\n``` \n\nDisplay this object in jupyter. \n\n### `get` \n\nView source \n\n```\n@classmethod\nget(\nclient, entity=None, project=None, sid=None, order=None, query=None, \\*\\*kwargs\n)\n\n``` \n\n### `load` \n\nView source", "matching_document_document_id": "22d1890a-6b8a-47e3-93e5-47ad68eb9c99", "query_weight": 0.6503225806451612 }, { "query_id": "ddb11209-eb35-4b1d-9f36-ea7423f31ade", "query": "wandb log_code method", "matching_document": "## Run\n### Methods\n#### `log_code`\n| Arguments | |\n| --- | --- |\n| `root` | The relative (to `os.getcwd()`) or absolute path to recursively find code from. |\n| `name` | (str, optional) The name of our code artifact. By default, we'll name the artifact `source-$PROJECT_ID-$ENTRYPOINT_RELPATH`. There may be scenarios where you want many runs to share the same artifact. Specifying name allows you to achieve that. |\n| `include_fn` | A callable that accepts a file path and (optionally) root path and returns True when it should be included and False otherwise. This defaults to: `lambda path, root: path.endswith(\".py\")` |\n| `exclude_fn` | A callable that accepts a file path and (optionally) root path and returns `True` when it should be excluded and `False` otherwise. This defaults to a function that excludes all files within `<root>/.wandb/` and `<root>/wandb/` directories. | \n\n#### Examples: \n\nBasic usage \n\n```\nrun.log\\_code()\n\n``` \n\nAdvanced usage", "matching_document_document_id": "1c1ab624-25b4-465c-ac9d-75d589ae2177", "query_weight": 0.1 }, { "query_id": "ac91b849-4d24-4915-91b3-ab8d1827d6a8", "query": "What are the compatible data types for wandb Table columns?", "matching_document": "## Table\n\nView source on GitHub \n\nThe Table class used to display and analyze tabular data. \n\n```\nTable(\ncolumns=None, data=None, rows=None, dataframe=None, dtype=None, optional=(True),\nallow\\_mixed\\_types=(False)\n)\n\n``` \n\nUnlike traditional spreadsheets, Tables support numerous types of data:\nscalar values, strings, numpy arrays, and most subclasses of `wandb.data_types.Media`.\nThis means you can embed `Images`, `Video`, `Audio`, and other sorts of rich, annotated media\ndirectly in Tables, alongside other traditional scalar values. \n\nThis class is the primary class used to generate the Table Visualizer\nin the UI: . \n\nTables can be constructed with initial data using the `data` or\n`dataframe` parameters:", "matching_document_document_id": "ef293fdf-142d-4a6e-b668-514932593e8f", "query_weight": 0.1 }, { "query_id": "1bb37389-6aa0-4cdf-8520-25bd43a1fe0f", "query": "W&B collaboration features for non-account holders", "matching_document": "## Anonymous Mode\n#### How does someone without an account see results?\n1. **Auto-create temporary account:** W&B checks for an account that's already signed in. If there's no account, we automatically create a new anonymous account and save that API key for the session.\n2. **Log results quickly:** The user can run and re-run the script, and automatically see results show up in the W&B dashboard UI. These unclaimed anonymous runs will be available for 7 days.\n3. **Claim data when it's useful**: Once the user finds valuable results in W&B, they can easily click a button in the banner at the top of the page to save their run data to a real account. If they don't claim a run, it will be deleted after 7 days. \n\n:::caution\n**Anonymous run links are sensitive**. These links allow anyone to view and claim the results of an experiment for 7 days, so make sure to only share links with people you trust. If you're trying to share results publicly, but hide the author's identity, please contact us at support@wandb.com to share more about your use case.\n:::", "matching_document_document_id": "9aa538cf-da7d-4859-b987-1c2503186a6c", "query_weight": 1.3470967741935482 }, { "query_id": "21a527ea-a245-4e99-b7cd-817667cc839b", "query": "Wandb YAML configuration", "matching_document": "## Add W&B to your code\n#### Training script with W&B Python SDK\nTo create a W&B Sweep, we first create a YAML configuration file. The configuration file contains he hyperparameters we want the sweep to explore. In the proceeding example, the batch size (`batch_size`), epochs (`epochs`), and the learning rate (`lr`) hyperparameters are varied during each sweep. \n\n```\n# config.yaml\nprogram: train.py\nmethod: random\nname: sweep\nmetric:\ngoal: maximize\nname: val\\_acc\nparameters:\nbatch\\_size:\nvalues: [16,32,64]\nlr:\nmin: 0.0001\nmax: 0.1\nepochs:\nvalues: [5, 10, 15]\n\n``` \n\nFor more information on how to create a W&B Sweep configuration, see Define sweep configuration. \n\nNote that you must provide the name of your Python script for the `program` key in your YAML file. \n\nNext, we add the following to the code example:", "matching_document_document_id": "77cd3e2d-cd4c-4d88-8ffb-2bc94492b051", "query_weight": 0.8825806451612903 }, { "query_id": "9386e746-d55d-439e-b3de-c891a9edd63e", "query": "How to correctly use markdown in Weights & Biases logs or reports?", "matching_document": "## Edit a report\n#### Markdown\nAdd markdown to your report interactively with the App UI or with the W&B SDK. \n\nEnter a forward slash (`/`) in the report to display a dropdown menu. From the dropdown choose **Markdown**. \n\nUse the `wandb.apis.reports.MarkdownBlock` Class to create a markdown block programmatically. Pass a string to the `text` parameter: \n\n```\nimport wandb\nimport wandb.apis.reports as wr\n\nreport = wr.Report(project=\"report-editing\")\n\nreport.blocks = [\nwr.MarkdownBlock(text=\"Markdown cell with \\*italics\\* and \\*\\*bold\\*\\* and $e=mc^2$\")\n]\n\n``` \n\nThis will render a markdown block similar to:", "matching_document_document_id": "c651c051-93c3-41b2-bfc8-844d8e467a7f", "query_weight": 0.1 }, { "query_id": "1ce58ba0-dd3c-4792-b9bb-034e12aeaa10", "query": "Weights & Biases code tracking", "matching_document": "## Run\n### Methods\n#### `log_code`\n| Arguments | |\n| --- | --- |\n| `root` | The relative (to `os.getcwd()`) or absolute path to recursively find code from. |\n| `name` | (str, optional) The name of our code artifact. By default, we'll name the artifact `source-$PROJECT_ID-$ENTRYPOINT_RELPATH`. There may be scenarios where you want many runs to share the same artifact. Specifying name allows you to achieve that. |\n| `include_fn` | A callable that accepts a file path and (optionally) root path and returns True when it should be included and False otherwise. This defaults to: `lambda path, root: path.endswith(\".py\")` |\n| `exclude_fn` | A callable that accepts a file path and (optionally) root path and returns `True` when it should be excluded and `False` otherwise. This defaults to a function that excludes all files within `<root>/.wandb/` and `<root>/wandb/` directories. | \n\n#### Examples: \n\nBasic usage \n\n```\nrun.log\\_code()\n\n``` \n\nAdvanced usage", "matching_document_document_id": "1c1ab624-25b4-465c-ac9d-75d589ae2177", "query_weight": 0.9754838709677419 }, { "query_id": "1420f94a-41ce-4c72-96a4-14c9616dc01a", "query": "Best practices for tracking model training in W&B", "matching_document": "## Track Experiments\n### How it works\nTrack a machine learning experiment with a few lines of code:\n1. Create a W&B run.\n2. Store a dictionary of hyperparameters, such as learning rate or model type, into your configuration (`wandb.config`).\n3. Log metrics (`wandb.log()`) over time in a training loop, such as accuracy and loss.\n4. Save outputs of a run, like the model weights or a table of predictions. \n\nThe proceeding pseudocode demonstrates a common W&B Experiment tracking workflow: \n\n```python showLineNumbers\n\n# 1. Start a W&B Run\n\nwandb.init(entity=\"\", project=\"my-project-name\")\n\n# 2. Save mode inputs and hyperparameters\n\nwandb.config.learning\\_rate = 0.01\n\n# Import model and data\n\nmodel, dataloader = get\\_model(), get\\_data()\n\n# Model training code goes here\n\n# 3. Log metrics over time to visualize performance\n\nwandb.log({\"loss\": loss})\n\n# 4. Log an artifact to W&B\n\nwandb.log\\_artifact(model)\n```", "matching_document_document_id": "1c7f8798-7b2a-4baa-9829-14ada61db6bc", "query_weight": 0.1 }, { "query_id": "b49c9dc3-0d86-469f-bc76-9f1db5ddf210", "query": "How to use W&B for experiment tracking and logging model names", "matching_document": "## Track Experiments\n### How it works\nTrack a machine learning experiment with a few lines of code:\n1. Create a W&B run.\n2. Store a dictionary of hyperparameters, such as learning rate or model type, into your configuration (`wandb.config`).\n3. Log metrics (`wandb.log()`) over time in a training loop, such as accuracy and loss.\n4. Save outputs of a run, like the model weights or a table of predictions. \n\nThe proceeding pseudocode demonstrates a common W&B Experiment tracking workflow: \n\n```python showLineNumbers\n\n# 1. Start a W&B Run\n\nwandb.init(entity=\"\", project=\"my-project-name\")\n\n# 2. Save mode inputs and hyperparameters\n\nwandb.config.learning\\_rate = 0.01\n\n# Import model and data\n\nmodel, dataloader = get\\_model(), get\\_data()\n\n# Model training code goes here\n\n# 3. Log metrics over time to visualize performance\n\nwandb.log({\"loss\": loss})\n\n# 4. Log an artifact to W&B\n\nwandb.log\\_artifact(model)\n```", "matching_document_document_id": "1c7f8798-7b2a-4baa-9829-14ada61db6bc", "query_weight": 0.1 }, { "query_id": "81be3b21-b3dd-41b5-a383-927136e729e0", "query": "W&B data logging capabilities", "matching_document": "## Managing and Tracking ML Experiments With W&B\n### Logging Advanced\u00a0Things\u200b\n\nOne of the coolest things about W&B is that you can literally log anything. You can log custom metrics, matplotlib plots, datasets, embeddings from your models, prediction distribution, etc. \n\nRecently, Weights & Biases announced the Tables feature, which allows you to log, query and analyze tabular data. You can even visualize model predictions and compare them across models. For example: see the image below (taken from[ W&B Docs](http://docs.wandb.ai)), which compares two segmentation models. \n\nYou can log audio data, images, histograms, text, video, and tabular data and visualize/inspect them interactively. To learn more about W&B Tables, go through their [documentation](https://docs.wandb.ai/guides/data-vis). \n\nYou can even export the dashboard in CSV files to analyze them further. W&B also supports exports in PNG, SVG, PDF, and CSV, depending on the type of data you are trying to export.", "matching_document_document_id": "5b04907e-a04e-40a7-939f-b745aa031559", "query_weight": 1.2541935483870967 }, { "query_id": "e2588122-0bbc-4947-ae71-224385a4978c", "query": "How do I start a project in WandB?", "matching_document": "## wandb init\n\n**Usage** \n\n`wandb init [OPTIONS]` \n\n**Summary** \n\nConfigure a directory with Weights & Biases \n\n**Options** \n\n| **Option** | **Description** |\n| --- | --- |\n| -p, --project | The project to use. |\n| -e, --entity | The entity to scope the project to. |\n| --reset | Reset settings |\n| -m, --mode | Can be \"online\", \"offline\" or \"disabled\". Defaults to online. |", "matching_document_document_id": "ab01e901-8752-4cc2-84e5-9921b115a621", "query_weight": 0.9290322580645161 }, { "query_id": "73e0f4e5-4b3c-46f5-a65d-856a266ba90d", "query": "Weights & Biases artifact storage solutions", "matching_document": "## Data\n\n## AWS S3, Google Cloud Storage and W&B Artifacts \n\nWeights and Biases **Artifacts** enable you to log end-to-end training pipelines to ensure your experiments are always reproducible. \n\nData privacy is critical to Weights & Biases and so we support the creation of Artifacts from reference locations such as your own private cloud such as AWS S3 or Google Cloud Storage. Local, on-premises of W&B are also available upon request. \n\nBy default, W&B stores artifact files in a private Google Cloud Storage bucket located in the United States. All files are encrypted at rest and in transit. For sensitive files, we recommend a private W&B installation or the use of reference artifacts. \n\n##\u00a0Artifacts Reference Example\n**Create an artifact with the S3/GCS metadata** \n\nThe artifact only consists of metadata about the S3/GCS object such as its ETag, size, and version ID (if object versioning is enabled on the bucket). \n\n## Login to W&B \n\nLogin to Weights and Biases \n\n## Vehicle Loan Dataset", "matching_document_document_id": "15a6bbe9-03fd-4f70-89f9-3a0ad32cc614", "query_weight": 0.5109677419354839 }, { "query_id": "3060ed64-eeda-41eb-8f04-cafd2ad0130d", "query": "Are there any common mistakes to avoid when calling wandb.finish() in a script?", "matching_document": "## Train model with DDP\nThe preceding image demonstrates the W&B App UI dashboard. On the sidebar we see two experiments. One labeled 'null' and a second (bound by a yellow box) called 'DPP'. If you expand the group (select the Group dropdown) you will see the W&B Runs that are associated to that experiment. \n\n### Use W&B Service to avoid common distributed training issues. \n\nThere are two common issues you might encounter when using W&B and distributed training: \n\n1. **Hanging at the beginning of training** - A `wandb` process can hang if the `wandb` multiprocessing interferes with the multiprocessing from distributed training.\n2. **Hanging at the end of training** - A training job might hang if the `wandb` process does not know when it needs to exit. Call the `wandb.finish()` API at the end of your Python script to tell W&B that the Run finished. The wandb.finish() API will finish uploading data and will cause W&B to exit. \n\n### Enable W&B Service \n\n### Example use cases for multiprocessing", "matching_document_document_id": "39684e44-a845-4e76-bafe-10b5fda1cf4b", "query_weight": 0.6967741935483871 }, { "query_id": "788cdc1c-54fc-4289-b500-efc25348f44e", "query": "wandb.init resume allow vs auto differences", "matching_document": "| `resume` | (bool, str, optional) Sets the resuming behavior. Options: `\"allow\"`, `\"must\"`, `\"never\"`, `\"auto\"` or `None`. Defaults to `None`. Cases: - `None` (default): If the new run has the same ID as a previous run, this run overwrites that data. - `\"auto\"` (or `True`): if the previous run on this machine crashed, automatically resume it. Otherwise, start a new run. - `\"allow\"`: if id is set with `init(id=\"UNIQUE_ID\")` or `WANDB_RUN_ID=\"UNIQUE_ID\"` and it is identical to a previous run, wandb will automatically resume the run with that id. Otherwise, wandb will start a new run. - `\"never\"`: if id is set with `init(id=\"UNIQUE_ID\")` or `WANDB_RUN_ID=\"UNIQUE_ID\"` and it is identical to a previous run, wandb will crash. - `\"must\"`: if id is set with `init(id=\"UNIQUE_ID\")` or `WANDB_RUN_ID=\"UNIQUE_ID\"` and it is identical to a previous run, wandb will automatically resume the run with the id. Otherwise, wandb will crash. See our guide to resuming runs for more. |\n| `reinit` | (bool, optional) Allow multiple `wandb.init()` calls in the same process. (default: `False`) |\n| `magic` | (bool, dict, or str, optional) The bool controls whether we try to auto-instrument your script, capturing basic details of your run without you having to add more wandb code. (default: `False`) You can also pass a dict, json string, or yaml filename. |\n| `config_exclude_keys` | (list, optional) string keys to exclude from `wandb.config`. |\n| `config_include_keys` | (list, optional) string keys to include in `wandb.config`. |", "matching_document_document_id": "01c9bdf0-fd26-4350-8de1-e75588077941", "query_weight": 1.0219354838709678 }, { "query_id": "6efc37af-4128-443b-9e55-73771bdd2f29", "query": "How to configure a sweep for parallel execution in Weights & Biases?", "matching_document": "## Sweeps on Launch\n### Create a sweep with a W&B standard scheduler\n1. Create a Sweep configuration\n2. Specify the full job name within your sweep configuration\n3. Initialize a sweep agent. \n\n:::info\nSteps 1 and 3 are the same steps you normally take when you create a W&B Sweep.\n::: \n\nFor example, in the following code snippet, we specify `'wandb/jobs/Hello World 2:latest'` for the job value: \n\n```\n# launch-sweep-config.yaml\n\njob: 'wandb/jobs/Hello World 2:latest'\ndescription: sweep examples using launch jobs\n\nmethod: bayes\nmetric:\ngoal: minimize\nname: loss\\_metric\nparameters:\nlearning\\_rate:\nmax: 0.02\nmin: 0\ndistribution: uniform\nepochs:\nmax: 20\nmin: 0\ndistribution: int\\_uniform\n\n# Optional scheduler parameters:\n\n# scheduler:\n# num\\_workers: 1 # concurrent sweep runs\n# docker\\_image: \n# resource: \n# resource\\_args: # resource arguments passed to runs\n# env:\n# - WANDB\\_API\\_KEY\n\n# Optional Launch Params\n# launch:\n# registry: \n\n```", "matching_document_document_id": "0d492067-2ad4-4bd1-aa57-a8cd5700fde5", "query_weight": 1.207741935483871 }, { "query_id": "85a95de3-72c1-4447-8b37-42888f87f34c", "query": "Can you run parallel instances of a sweep across different GPUs in Weights & Biases?", "matching_document": "## Parallelize agents\n#### Parallelize on a multi-GPU machine\n1. Open more than one terminal window on your local machine.\n2. Specify the GPU instance to use with `CUDA_VISIBLE_DEVICES` when you start a W&B Sweep job (`wandb agent`). Assign `CUDA_VISIBLE_DEVICES` an integer value corresponding to the GPU instance to use. \n\nFor example, suppose you have two NVIDIA GPUs on your local machine. Open a terminal window and set `CUDA_VISIBLE_DEVICES` to `0` (`CUDA_VISIBLE_DEVICES=0`). Replace `sweep_ID` in the proceeding example with the W&B Sweep ID that is returned when you initialized a W&B Sweep: \n\nTerminal 1 \n\n```\nCUDA\\_VISIBLE\\_DEVICES=0 wandb agent sweep_ID\n\n``` \n\nOpen a second terminal window. Set `CUDA_VISIBLE_DEVICES` to `1` (`CUDA_VISIBLE_DEVICES=1`). Paste the same W&B Sweep ID for the `sweep_ID` mentioned in the proceeding code snippet: \n\nTerminal 2 \n\n```\nCUDA\\_VISIBLE\\_DEVICES=1 wandb agent sweep_ID\n\n```", "matching_document_document_id": "03228a82-a0d9-405b-ba85-e645187827e1", "query_weight": 1.207741935483871 }, { "query_id": "5fbdd25f-338d-4b72-8682-879bd42c8e02", "query": "How to start multiple WANDB runs in parallel?", "matching_document": "## Experiments FAQ\n\nFrequently Asked Questions About Experiments \n\nThe proceeding questions are commonly asked questions about W&B Artifacts. \n\n### How do I launch multiple runs from one script? \n\nUse `wandb.init` and `run.finish()` to log multiple Runs from one script: \n\n1. `run = wandb.init(reinit=True)`: Use this setting to allow reinitializing runs\n2. `run.finish()`: Use this at the end of your run to finish logging for that run \n\n```\nimport wandb\n\nfor x in range(10):\nrun = wandb.init(reinit=True)\nfor y in range(100):\nwandb.log({\"metric\": x + y})\nrun.finish()\n\n``` \n\nAlternatively you can use a python context manager which will automatically finish logging: \n\n```\nimport wandb\n\nfor x in range(10):\nrun = wandb.init(reinit=True)\nwith run:\nfor y in range(100):\nrun.log({\"metric\": x + y})\n\n``` \n\n### `InitStartError: Error communicating with wandb process` \n\nThis error indicates that the library is having difficulty launching the process which synchronizes data to the server.", "matching_document_document_id": "9b1af9fa-cbac-449f-a522-8d9199665bcf", "query_weight": 0.8361290322580645 }, { "query_id": "d67da893-a188-49a5-a8f0-b706f6a703c1", "query": "Resuming training runs with Weights & Biases using saved checkpoints", "matching_document": "## Use PyTorch Lightning with Weights & Biases\n### Visualizing Performance with Weights & Biases\nBeing able to save and restore models also allows you to collaborate more effectively with your team and return to experiments from a few weeks ago. \n\nTo save PyTorch lightning models with Weights & Biases, we use: \n\n```\ntrainer.save_checkpoint('EarlyStoppingADam-32-0.001.pth')\nwandb.save('EarlyStoppingADam-32-0.001.pth')\n``` \n\nThis creates a checkpoint file in the local runtime and uploads it to W&B. Now, when we decide to resume training even on a different system, we can simply load the checkpoint file from W&B and load it into our program like so: \n\n```\nwandb.restore('EarlyStoppingADam-32-0.001.pth')\nmodel.load_from_checkpoint('EarlyStoppingADam-32-0.001.pth')\n``` \n\nNow the checkpoint has been loaded into the model and the training can be resumed using the desired training module.", "matching_document_document_id": "a17a29b5-7f61-4cbc-83b6-fbbf4cd1d550", "query_weight": 0.5574193548387096 }, { "query_id": "07afdd0a-ae3c-47bb-83ba-f6e48e71d164", "query": "Weights & Biases API documentation", "matching_document": "## Api\n\nView source on GitHub \n\nUsed for querying the wandb server. \n\n```\nApi(\noverrides=None,\ntimeout: Optional[int] = None,\napi\\_key: Optional[str] = None\n) -> None\n\n``` \n\n#### Examples: \n\nMost common way to initialize \n\n```\n>>> wandb.Api()\n\n``` \n\n| Arguments | |\n| --- | --- |\n| `overrides` | (dict) You can set `base_url` if you are using a wandb server other than . You can also set defaults for `entity`, `project`, and `run`. | \n\n| Attributes | |\n| --- | --- |\n| | | \n\n## Methods \n\n### `artifact` \n\nView source \n\n```\nartifact(\nname, type=None\n)\n\n``` \n\nReturn a single artifact by parsing path in the form `entity/project/name`. \n\n| Arguments | |\n| --- | --- |\n| `name` | (str) An artifact name. May be prefixed with entity/project. Valid names can be in the following forms: name:version name:alias |\n| `type` | (str, optional) The type of artifact to fetch. | \n\n| Returns | |\n| --- | --- |\n| A `Artifact` object. | | \n\n### `artifact_type` \n\nView source \n\n### `flush`", "matching_document_document_id": "8d4ccf38-8646-42b5-abfa-b46e0c4ca9f5", "query_weight": 0.1 }, { "query_id": "2125b951-8859-40be-b6f3-cbc563e6ba40", "query": "Best practices for model versioning in W&B", "matching_document": "## \u270d\ufe0f W&B Best Practices\n\n1. **Projects**: Log multiple runs to a project to compare them. `wandb.init(project=\"project-name\")`\n2. **Groups**: For multiple processes or cross validation folds, log each process as a runs and group them together. `wandb.init(group=\"experiment-1\")`\n3. **Tags**: Add tags to track your current baseline or production model.\n4. **Notes**: Type notes in the table to track the changes between runs.\n5. **Reports**: Take quick notes on progress to share with colleagues and make dashboards and snapshots of your ML projects.", "matching_document_document_id": "aa2dbfa2-471e-4ec7-b1a2-60a608aa6873", "query_weight": 0.1 }, { "query_id": "b56ba171-e8c4-4ecb-b252-707d2f6fef5b", "query": "Deep Learning project tracking with W&B", "matching_document": "## Track Experiments\n### How it works\nTrack a machine learning experiment with a few lines of code:\n1. Create a W&B run.\n2. Store a dictionary of hyperparameters, such as learning rate or model type, into your configuration (`wandb.config`).\n3. Log metrics (`wandb.log()`) over time in a training loop, such as accuracy and loss.\n4. Save outputs of a run, like the model weights or a table of predictions. \n\nThe proceeding pseudocode demonstrates a common W&B Experiment tracking workflow: \n\n```python showLineNumbers\n\n# 1. Start a W&B Run\n\nwandb.init(entity=\"\", project=\"my-project-name\")\n\n# 2. Save mode inputs and hyperparameters\n\nwandb.config.learning\\_rate = 0.01\n\n# Import model and data\n\nmodel, dataloader = get\\_model(), get\\_data()\n\n# Model training code goes here\n\n# 3. Log metrics over time to visualize performance\n\nwandb.log({\"loss\": loss})\n\n# 4. Log an artifact to W&B\n\nwandb.log\\_artifact(model)\n```", "matching_document_document_id": "1c7f8798-7b2a-4baa-9829-14ada61db6bc", "query_weight": 1.2541935483870967 }, { "query_id": "19166e74-3631-4559-9abe-af00bd4f6844", "query": "best practices for using Weights & Biases in AI projects", "matching_document": "## \u270d\ufe0f W&B Best Practices\n\n1. **Projects**: Log multiple runs to a project to compare them. `wandb.init(project=\"project-name\")`\n2. **Groups**: For multiple processes or cross validation folds, log each process as a runs and group them together. `wandb.init(group=\"experiment-1\")`\n3. **Tags**: Add tags to track your current baseline or production model.\n4. **Notes**: Type notes in the table to track the changes between runs.\n5. **Reports**: Take quick notes on progress to share with colleagues and make dashboards and snapshots of your ML projects.", "matching_document_document_id": "aa2dbfa2-471e-4ec7-b1a2-60a608aa6873", "query_weight": 1.2541935483870967 }, { "query_id": "55101186-d819-4023-9935-2e0d4db6915f", "query": "Handling large datasets", "matching_document": "## Plotting with Pandas and Weights & Biases: Step-by-Step Guide\n### Challenges and Considerations\n\n### Handling Large Datasets and Performance Optimization \n\nHandling large datasets using Pandas can sometimes land you in trouble. There are some general guidelines you can follow to get the most out of Pandas \n\n* Try to pre-process your dataset in smaller batches and then merge it into a single dataframe\n* Consider writing Cython for Pandas\n* Use the \"numba\" backend to utilize JIT (Just-in-Time) compilation \n\nThere are other libraries that you can also consider, such as [dask](https://www.dask.org/), [cudf,](https://docs.rapids.ai/api/cudf/stable/) and [polars](https://github.com/pola-rs/polars) which boast better performances than Pandas. \n\n### Addressing Missing or Inconsistent Data for Accurate Plotting \n\nHandling missing and inconsistent data is an important skill one must have. Pandas provide us with a number of utility functions to deal with them.", "matching_document_document_id": "d8a55b89-a3f3-4f36-ab15-1f4036bf39ef", "query_weight": 2.0903225806451613 }, { "query_id": "5f7390ee-de7c-44f7-bc3e-69df6fd2a216", "query": "WandbLogger log_image method usage examples", "matching_document": "## PyTorch Lightning\n### Using PyTorch Lightning's `WandbLogger`\n#### Log images, text and more\n\nThe `WandbLogger` has `log_image`, `log_text` and `log_table` methods for logging media. \n\nYou can also directly call `wandb.log` or `trainer.logger.experiment.log` to log other media types such as Audio, Molecules, Point Clouds, 3D Objects and more. \n\n```\n# using tensors, numpy arrays or PIL images\nwandb\\_logger.log\\_image(key=\"samples\", images=[img1, img2])\n\n# adding captions\nwandb\\_logger.log\\_image(key=\"samples\", images=[img1, img2], caption=[\"tree\", \"person\"])\n\n# using file path\nwandb\\_logger.log\\_image(key=\"samples\", images=[\"img\\_1.jpg\", \"img\\_2.jpg\"])\n\n# using .log in the trainer\ntrainer.logger.experiment.log(\n{\"samples\": [wandb.Image(img, caption=caption) for (img, caption) in my\\_images]},\nstep=current\\_trainer\\_global\\_step,\n)\n\n```", "matching_document_document_id": "d0659f56-4719-4d64-8ef3-6eee597ac42a", "query_weight": 0.9754838709677419 }, { "query_id": "0ef41cda-5184-4aac-aa8f-33205af2d0e4", "query": "How to use custom YAML config files with Wandb", "matching_document": "## Configure Experiments\n### File-Based Configs\nKey-value pairs are automatically passed to `wandb.config` if you create a file called `config-defaults.yaml`. \n\nThe proceeding code snippet demonstrates a sample `config-defaults.yaml` YAML file: \n\n```\n# config-defaults.yaml\n# sample config defaults file\nepochs:\ndesc: Number of epochs to train over\nvalue: 100\nbatch\\_size:\ndesc: Size of each mini-batch\nvalue: 32\n\n``` \n\nYou can overwrite automatically passed by a `config-defaults.yaml`. To do so , pass values to the `config` argument of `wandb.init`. \n\nYou can also load different config files with the command line argument `--configs`. \n\n### Example use case for file-based configs \n\nSuppose you have a YAML file with some metadata for the run, and then a dictionary of hyperparameters in your Python script. You can save both in the nested `config` object:", "matching_document_document_id": "93f4f4c2-1aa0-4d0b-b7e8-171f2df8b8f3", "query_weight": 0.8825806451612903 }, { "query_id": "2e76200c-5d5d-45d0-87ca-b0c10a36f70e", "query": "Weights & Biases image logging", "matching_document": "\"\"\"Format images for logging to W&B.\n\n Arguments:\n data_or_path: (numpy array, string, io) Accepts numpy array of\n image data, or a PIL image. The class attempts to infer\n the data format and converts it.\n mode: (string) The PIL mode for an image. Most common are \"L\", \"RGB\",\n \"RGBA\". Full explanation at https://pillow.readthedocs.io/en/stable/handbook/concepts.html#modes\n caption: (string) Label for display of image.\n\n Note : When logging a `torch.Tensor` as a `wandb.Image`, images are normalized. If you do not want to normalize your images, please convert your tensors to a PIL Image.\n\n Examples:\n ### Create a wandb.Image from a numpy array\n \n ```python\n import numpy as np\n import wandb\n\n with wandb.init() as run:\n examples = []\n for i in range(3):\n pixels = np.random.randint(low=0, high=256, size=(100, 100, 3))\n image = wandb.Image(pixels, caption=f\"random field {i}\")\n examples.append(image)\n run.log({\"examples\": examples})\n ```", "matching_document_document_id": "d99652d5-58ec-4116-995f-69388c97277f", "query_weight": 1.8116129032258064 }, { "query_id": "fde70568-cb7e-481a-9ee7-df3a84ef24b2", "query": "Explain how I can version datasets with Weights & Biases. How do I create a new version of a dataset? How does Weights & Biases work with versions? What are some best practices for versioning datasets?", "matching_document": "## Dataset Versioning\n### Core Artifacts features\n1. **Upload**: Start tracking and versioning any data (files or directories) with `run.log_artifact()`. You can also track datasets in a remote filesystem (e.g. cloud storage in S3 or GCP) by reference, using a link or URI instead of the raw contents.\n2. **Version**: Define an artifact by giving it a type (`\"raw_data\"`, `\"preprocessed_data\"`, `\"balanced_data\"`) and a name (`\"imagenet_cats_10K\"`). When you log the same name again, W&B automatically creates a new version of the artifact with the latest contents.\n3. **Alias**: Set an alias like `\"best\"` or `\"production\"` to highlight the important versions in a lineage of artifacts.\n4. **Compare**: Select any two versions to browse the contents side-by-side. We're also working on a tool for dataset visualization, learn more here.\n5. **Download**: Obtain a local copy of the artifact or verify the contents by reference. \n\nFor more detail on these features, check out Artifacts how it works.", "matching_document_document_id": "7407e418-a665-438f-b605-d5dc12e4240e", "query_weight": 1.3006451612903225 }, { "query_id": "dc20f4da-35a8-4529-af52-ac46dea0f41a", "query": "How does Weights & Biases integrate with S3 versioning features?", "matching_document": "## Dataset Versioning\n### Core Artifacts features\n1. **Upload**: Start tracking and versioning any data (files or directories) with `run.log_artifact()`. You can also track datasets in a remote filesystem (e.g. cloud storage in S3 or GCP) by reference, using a link or URI instead of the raw contents.\n2. **Version**: Define an artifact by giving it a type (`\"raw_data\"`, `\"preprocessed_data\"`, `\"balanced_data\"`) and a name (`\"imagenet_cats_10K\"`). When you log the same name again, W&B automatically creates a new version of the artifact with the latest contents.\n3. **Alias**: Set an alias like `\"best\"` or `\"production\"` to highlight the important versions in a lineage of artifacts.\n4. **Compare**: Select any two versions to browse the contents side-by-side. We're also working on a tool for dataset visualization, learn more here.\n5. **Download**: Obtain a local copy of the artifact or verify the contents by reference. \n\nFor more detail on these features, check out Artifacts how it works.", "matching_document_document_id": "7407e418-a665-438f-b605-d5dc12e4240e", "query_weight": 1.3006451612903225 }, { "query_id": "c3bd916e-d6a9-44e8-a0cc-6c3af994c22f", "query": "W&B custom objects logging", "matching_document": "## log\nLogged values don't have to be scalars. Logging any wandb object is supported.\nFor example `wandb.log({\"example\": wandb.Image(\"myimage.jpg\")})` will log an\nexample image which will be displayed nicely in the W&B UI.\nSee the reference documentation\nfor all of the different supported types or check out our\nguides to logging for examples,\nfrom 3D molecular structures and segmentation masks to PR curves and histograms.\n`wandb.Table`s can be used to logged structured data. See our\nguide to logging tables\nfor details. \n\nLogging nested metrics is encouraged and is supported in the W&B UI.\nIf you log with a nested dictionary like `wandb.log({\"train\": {\"acc\": 0.9}, \"val\": {\"acc\": 0.8}})`, the metrics will be organized into\n`train` and `val` sections in the W&B UI.", "matching_document_document_id": "74357cdb-b4f7-4b1c-8aab-2c844ea640de", "query_weight": 1.2541935483870967 }, { "query_id": "717c3817-6e7b-4613-96d1-ec17fb2d1c8b", "query": "Data type compatibility in wandb Table and pandas DataFrame", "matching_document": "## Table\n```\nimport pandas as pd\nimport wandb\n\ndata = {\"users\": [\"geoff\", \"juergen\", \"ada\"], \"feature\\_01\": [1, 117, 42]}\ndf = pd.DataFrame(data)\n\ntbl = wandb.Table(data=df)\nassert all(tbl.get\\_column(\"users\") == df[\"users\"])\nassert all(tbl.get\\_column(\"feature\\_01\") == df[\"feature\\_01\"])\n\n``` \n\nAdditionally, users can add data to Tables incrementally by using the\n`add_data`, `add_column`, and `add_computed_column` functions for\nadding rows, columns, and columns computed from data in other columns, respectively: \n\n```\nimport wandb\n\ntbl = wandb.Table(columns=[\"user\"])\n\nusers = [\"geoff\", \"juergen\", \"ada\"]\n\n[tbl.add\\_data(user) for user in users]\nassert tbl.get\\_column(\"user\") == users\n\ndef get\\_user\\_name\\_length(index, row):\nreturn {\"feature\\_01\": len(row[\"user\"])}\n\ntbl.add\\_computed\\_columns(get\\_user\\_name\\_length)\nassert tbl.get\\_column(\"feature\\_01\") == [5, 7, 3]\n\n``` \n\nTables can be logged directly to runs using `run.log({\"my_table\": table})`\nor added to artifacts using `artifact.add(table, \"my_table\")`:", "matching_document_document_id": "b915a272-8ca7-4a47-bb32-617e7d36e7ba", "query_weight": 0.8361290322580645 }, { "query_id": "5b63983d-9a21-481d-9a82-80ae594086dd", "query": "Optuna hyperparameter sweep management with Weights & Biases", "matching_document": "## Sweeps on Launch\n### Create a custom sweep scheduler\n```\n# optuna\\_config\\_basic.yaml\ndescription: A basic Optuna scheduler\njob: wandb/sweep-jobs/job-fashion-MNIST-train:latest\nrun\\_cap: 5\nmetric:\nname: epoch/val\\_loss\ngoal: minimize\n\nscheduler:\njob: wandb/sweep-jobs/job-optuna-sweep-scheduler:latest\nresource: local-container # required for scheduler jobs sourced from images\nnum\\_workers: 2\n\n# optuna specific settings\nsettings:\npruner:\ntype: PercentilePruner\nargs:\npercentile: 25.0 # kill 75% of runs\nn\\_warmup\\_steps: 10 # pruning disabled for first x steps\n\nparameters:\nlearning\\_rate:\nmin: 0.0001\nmax: 0.1\n``` \n\n3. Lastly, launch the sweep to an active queue with the launch-sweep command: \n\n```bash\nwandb launch-sweep -q -p -e \n``` \n\nFor the exact implementation of the Optuna sweep scheduler job, see wandb/launch-jobs. For more examples of what is possible with the Optuna scheduler, check out wandb/examples. \n\n\n", "matching_document_document_id": "dbd99861-821a-4f36-8add-fc595a861549", "query_weight": 0.1 }, { "query_id": "4b87323b-3ea9-47e4-83c6-8b32807ad64d", "query": "link run to model registry", "matching_document": "## W&B Tutorial with Pytorch Lightning\n### Model Registry\n\nAfter logging a bunch of checkpoints across multiple runs during experimentation, now comes time to hand-off the best checkpoint to the next stage of the workflow (e.g. testing, deployment). \n\nThe model registry offers a centralized place to house the best checkpoints for all your model tasks. Any `model` artifact you log can be \"linked\" to a Registered Model. Here are the steps to start using the model registry for more organized model management:\n1. Access your team's model registry by going the team page and selecting `Model Registry` \n\n1. Create a new Registered Model.\n2. Go to the artifacts tab of the project that holds all your model checkpoints\n3. Click \"Link to Registry\" for the model artifact version you want. (Alternatively you can link a model via api with `wandb.run.link_artifact`)", "matching_document_document_id": "cf34b579-6181-4435-87ca-06d3cee2dc38", "query_weight": 0.1 }, { "query_id": "8341e234-d080-4717-87e0-85e2959333bb", "query": "How to set constraints on hyperparameters in W&B sweeps?", "matching_document": "## Quickstart\n### Define the search space with a sweep configuration\nWithin a dictionary, specify what hyperparameters you want to sweep over and. For more information about configuration options, see Define sweep configuration. \n\nThe proceeding example demonstrates a sweep configuration that uses a random search (`'method':'random'`). The sweep will randomly select a random set of values listed in the configuration for the batch size, epoch, and the learning rate. \n\nThroughout the sweeps, W&B will maximize the metric specified in the metric key (`metric`). In the following example, W&B will maximize (`'goal':'maximize'`) the validation accuracy (`'val_acc'`). \n\n```\n# 2: Define the search space\nsweep\\_configuration = {\n'method': 'random',\n'metric': {'goal': 'minimize', 'name': 'score'},\n'parameters':\n{\n'x': {'max': 0.1, 'min': 0.01},\n'y': {'values': [1, 3, 7]},\n}\n}\n\n```", "matching_document_document_id": "7dea52bc-7acf-4a8b-9f5d-07d5b0e8e240", "query_weight": 0.8825806451612903 }, { "query_id": "9d9f15b0-8089-4485-9a0a-6c28955b3ea2", "query": "Best practices for managing code versioning with wandb artifacts", "matching_document": "--- \n\n## displayed\\_sidebar: default \n\n# Code Saving \n\nBy default, we only save the latest git commit hash. You can turn on more code features to compare the code between your experiments dynamically in the UI. \n\nStarting with `wandb` version 0.8.28, we can save the code from your main training file where you call `wandb.init()`. This will get sync'd to the dashboard and show up in a tab on the run page, as well as the Code Comparer panel. Go to your settings page to enable code saving by default. \n\n## Save Library Code \n\nWhen code saving is enabled, wandb will save the code from the file that called `wandb.init()`. To save additional library code, you have two options: \n\n* Call `wandb.run.log_code(\".\")` after calling `wandb.init()`\n* Pass a settings object to `wandb.init` with code\\_dir set: `wandb.init(settings=wandb.Settings(code_dir=\".\"))` \n\n## Code Comparer \n\n## Jupyter Session History \n\n## Jupyter diffing", "matching_document_document_id": "10c3ed4d-db30-4f77-b9c9-d90af6894203", "query_weight": 0.1 }, { "query_id": "3d3f0a73-87bf-4ed6-b357-e945edd9005c", "query": "Best practices for dataset versioning in Weights & Biases", "matching_document": "## Dataset Versioning\n### Core Artifacts features\n1. **Upload**: Start tracking and versioning any data (files or directories) with `run.log_artifact()`. You can also track datasets in a remote filesystem (e.g. cloud storage in S3 or GCP) by reference, using a link or URI instead of the raw contents.\n2. **Version**: Define an artifact by giving it a type (`\"raw_data\"`, `\"preprocessed_data\"`, `\"balanced_data\"`) and a name (`\"imagenet_cats_10K\"`). When you log the same name again, W&B automatically creates a new version of the artifact with the latest contents.\n3. **Alias**: Set an alias like `\"best\"` or `\"production\"` to highlight the important versions in a lineage of artifacts.\n4. **Compare**: Select any two versions to browse the contents side-by-side. We're also working on a tool for dataset visualization, learn more here.\n5. **Download**: Obtain a local copy of the artifact or verify the contents by reference. \n\nFor more detail on these features, check out Artifacts how it works.", "matching_document_document_id": "7407e418-a665-438f-b605-d5dc12e4240e", "query_weight": 1.3006451612903225 }, { "query_id": "a658de02-ad3c-4a7b-91f5-fcf484a2a884", "query": "running sweeps on multiple GPUs", "matching_document": "## Parallelize agents\n#### Parallelize on a multi-GPU machine\n1. Open more than one terminal window on your local machine.\n2. Specify the GPU instance to use with `CUDA_VISIBLE_DEVICES` when you start a W&B Sweep job (`wandb agent`). Assign `CUDA_VISIBLE_DEVICES` an integer value corresponding to the GPU instance to use. \n\nFor example, suppose you have two NVIDIA GPUs on your local machine. Open a terminal window and set `CUDA_VISIBLE_DEVICES` to `0` (`CUDA_VISIBLE_DEVICES=0`). Replace `sweep_ID` in the proceeding example with the W&B Sweep ID that is returned when you initialized a W&B Sweep: \n\nTerminal 1 \n\n```\nCUDA\\_VISIBLE\\_DEVICES=0 wandb agent sweep_ID\n\n``` \n\nOpen a second terminal window. Set `CUDA_VISIBLE_DEVICES` to `1` (`CUDA_VISIBLE_DEVICES=1`). Paste the same W&B Sweep ID for the `sweep_ID` mentioned in the proceeding code snippet: \n\nTerminal 2 \n\n```\nCUDA\\_VISIBLE\\_DEVICES=1 wandb agent sweep_ID\n\n```", "matching_document_document_id": "03228a82-a0d9-405b-ba85-e645187827e1", "query_weight": 1.207741935483871 }, { "query_id": "fef4ee28-9ebf-4bc9-9625-53bd493b57e0", "query": "save code as artifact wandb", "matching_document": "## Run\n### Methods\n#### `log_code`\n| Arguments | |\n| --- | --- |\n| `root` | The relative (to `os.getcwd()`) or absolute path to recursively find code from. |\n| `name` | (str, optional) The name of our code artifact. By default, we'll name the artifact `source-$PROJECT_ID-$ENTRYPOINT_RELPATH`. There may be scenarios where you want many runs to share the same artifact. Specifying name allows you to achieve that. |\n| `include_fn` | A callable that accepts a file path and (optionally) root path and returns True when it should be included and False otherwise. This defaults to: `lambda path, root: path.endswith(\".py\")` |\n| `exclude_fn` | A callable that accepts a file path and (optionally) root path and returns `True` when it should be excluded and `False` otherwise. This defaults to a function that excludes all files within `<root>/.wandb/` and `<root>/wandb/` directories. | \n\n#### Examples: \n\nBasic usage \n\n```\nrun.log\\_code()\n\n``` \n\nAdvanced usage", "matching_document_document_id": "1c1ab624-25b4-465c-ac9d-75d589ae2177", "query_weight": 0.7432258064516128 }, { "query_id": "bdea5082-75ac-49a0-b3c0-04a8bbc9b91c", "query": "wandb troubleshooting", "matching_document": "--- \n\n## displayed\\_sidebar: default \n\n# Troubleshooting \n\n### If wandb crashes, will it possibly crash my training run? \n\nIt is extremely important to us that we never interfere with your training runs. We run wandb in a separate process to make sure that if wandb somehow crashes, your training will continue to run. If the internet goes out, wandb will continue to retry sending data to wandb.ai. \n\n### Why is a run marked crashed in W&B when it\u2019s training fine locally? \n\nThis is likely a connection problem \u2014 if your server loses internet access and data stops syncing to W&B, we mark the run as crashed after a short period of retrying. \n\n### Does logging block my training? \n\n\"Is the logging function lazy? I don't want to be dependent on the network to send the results to your servers and then carry on with my local operations.\" \n\n### How do I stop wandb from writing to my terminal or my jupyter notebook output? \n\nSet the environment variable `WANDB_SILENT` to `true`.", "matching_document_document_id": "478fd537-67ea-43c0-bed4-c26eb6f93142", "query_weight": 1.1148387096774193 }, { "query_id": "0b0d58db-b315-403d-a7a5-ac05e1bff5e7", "query": "How to configure Weights & Biases to resume a run from the last saved checkpoint?", "matching_document": "## Train\n### Resuming Training\n1. Initialize the last wandb run by passing the `run id` from your Weights & Biases workspace to `wandb.init`\n2. Download the lastest checkpoint using `wandb.artifact`.\n3. Reinitialize the trainer and pass the `artifact_dir` to the `resume_from_checkpoint` argument in the `trainer.train` method. \n\n**Note: Change the `last_run_id` in the below cell to the id from your wandb run`** \n\n```\nlast_run_id = \"25d6hznl\" # fetch the run_id from your wandb workspace\n\n# resume the wandb run from the run_id\nrun = wandb.init(\nproject=os.environ[\"WANDB_PROJECT\"],\nid=last_run_id,\nresume=\"must\",\n)\n\n``` \n\n**Note: Change the `latest_checpoint`in the below cell to the checkpoint artifact from your run** \n\n```\n# fetch the checkpoint artifact from the run\n# this is usually in the format \"/checkpoint-:\nlatest_checkpoint = f\"parambharat/hf_transformers/checkpoint-{last_run_id}:v5\"\nartifact = run.use_artifact(latest_checkpoint, type=\"model\")\nartifact_dir = artifact.download()\n\n```", "matching_document_document_id": "e0736cb1-7216-49cc-8f68-5d6c92dff9f1", "query_weight": 0.5574193548387096 }, { "query_id": "b063e4f6-9696-4001-a81b-727c0b241915", "query": "Setup Weights & Biases for personal and company projects", "matching_document": "## \ud83d\ude80 Setup\n\nStart out by installing the experiment tracking library and setting up your free W&B account: \n\n1. Install with `!pip install`\n2. `import` the library into Python\n3. `.login()` so you can log metrics to your projects \n\nIf you've never used Weights & Biases before,\nthe call to `login` will give you a link to sign up for an account.\nW&B is free to use for personal and academic projects! \n\n```\n!pip install wandb -Uq\n\n``` \n\n```\nimport wandb\n\n``` \n\n```\nwandb.login()\n\n```", "matching_document_document_id": "ee82f3a4-c71a-434e-a6fe-0170efcb2674", "query_weight": 0.1 }, { "query_id": "24c71d4f-e63b-4f1d-b4b7-9c6fbca9098c", "query": "Access W&B tables in Python", "matching_document": "## Visualizing the Results Interactively as a W&B Table\n\nW&B Tables is an awesome Weights & Biases feature that lets interactively visualize and explore tabular data. And it is extremely easy to create one. \n\nAll we need to do is define a Pandas DataFrame (basically, defining a table in Python) with the four relevant to us columns. \n\n> \n\n```\nimport pandas as pd\n\npositive = predictions[:, 0].tolist()\nnegative = predictions[:, 1].tolist()\nneutral = predictions[:, 2].tolist()\n\ntable = {'Headline':headlines_list,\n\"Positive\":positive,\n\"Negative\":negative,\n\"Neutral\":neutral}\n\ndf = pd.DataFrame(table, columns = [\"Headline\", \"Positive\", \"Negative\", \"Neutral\"])\n\ndf.head(5)\n``` \n\n### Logging a W&B Table \n\nOkay. We're just 5 lines of code away from logging a W&B Table! The first thing we do is pip install and import the W&B library. \n\n```\n!pip install wandb\nimport wandb\n```", "matching_document_document_id": "5d634218-ff45-45d4-93ee-1495af0fa478", "query_weight": 1.2541935483870967 }, { "query_id": "ab2c9e11-eb02-4178-b865-7e8a89c4644e", "query": "log prompts Weights & Biases", "matching_document": "**Weights & Biases Prompts** is a suite of LLMOps tools built for the development of LLM-powered applications. \n\nUse W&B Prompts to visualize and inspect the execution flow of your LLMs, analyze the inputs and outputs of your LLMs, view the intermediate results and securely store and manage your prompts and LLM chain configurations. \n\n#### \ud83e\ude84 View Prompts In Action \n\n**In this notebook we will demostrate W&B Prompts:** \n\n* Using our 1-line LangChain integration\n* Using our Trace class when building your own LLM Pipelines \n\nSee here for the full W&B Prompts documentation \n\n## Installation \n\n```\n!pip install \"wandb>=0.15.4\" -qqq\n!pip install \"langchain>=0.0.218\" openai -qqq\n\n``` \n\n```\nimport langchain\nassert langchain.__version__ >= \"0.0.218\", \"Please ensure you are using LangChain v0.0.188 or higher\"\n\n``` \n\n## Setup \n\nThis demo requires that you have an OpenAI key \n\n# W&B Prompts \n\nW&B Prompts consists of three main components: \n\n**Trace table**: Overview of the inputs and outputs of a chain.", "matching_document_document_id": "5ccb89d6-6353-4b96-9920-060f32aad458", "query_weight": 1.8116129032258064 }, { "query_id": "bdf5f448-a43f-4718-a693-012c8830b0bf", "query": "Sharing Weights & Biases projects with team members", "matching_document": "--- \n\n## description: Collaborate and share W&B Reports with peers, co-workers, and your team. \n\n# Collaborate on reports \n\nCollaborate and Share W&B Reports \n\nOnce you have saved a report, you can select the **Share** button to collaborate. A draft copy of the report is created when you select the **Edit** button. Draft reports auto-save. Select **Save to report** to publish your changes to the shared report. \n\nA warning notification will appear if an edit conflict occurs. This can occur if you and another collaborator edit the same report at the same time. The warning notification will guide you to resolve potential edit conflicts. \n\n### Comment on reports \n\nClick the comment button on a panel in a report to add a comment directly to that panel. \n\n### Who can edit and share reports? \n\nReports that are created within an individual's private project is only visible to that user. The user can share their project to a team or to the public.", "matching_document_document_id": "7fe941de-f8fe-4acc-b460-004c8b6a298e", "query_weight": 2.136774193548387 }, { "query_id": "a535d678-2bc3-4706-9d0a-8a93a89c7849", "query": "What are the compatible data types for wandb Table columns when constructing from a pandas DataFrame?", "matching_document": "## Table\n\nView source on GitHub \n\nThe Table class used to display and analyze tabular data. \n\n```\nTable(\ncolumns=None, data=None, rows=None, dataframe=None, dtype=None, optional=(True),\nallow\\_mixed\\_types=(False)\n)\n\n``` \n\nUnlike traditional spreadsheets, Tables support numerous types of data:\nscalar values, strings, numpy arrays, and most subclasses of `wandb.data_types.Media`.\nThis means you can embed `Images`, `Video`, `Audio`, and other sorts of rich, annotated media\ndirectly in Tables, alongside other traditional scalar values. \n\nThis class is the primary class used to generate the Table Visualizer\nin the UI: . \n\nTables can be constructed with initial data using the `data` or\n`dataframe` parameters:", "matching_document_document_id": "ef293fdf-142d-4a6e-b668-514932593e8f", "query_weight": 0.8361290322580645 }, { "query_id": "0c8c3ea1-a08b-4565-893e-1e1d767ba5a3", "query": "Pandas DataFrame from W&B tables", "matching_document": "def _get_updated_result_type(self, row):\n \"\"\"Returns the updated result type based on the inputted row.\n\n Raises:\n TypeError: if the assignment is invalid.\n \"\"\"\n incoming_row_dict = {\n col_key: row[ndx] for ndx, col_key in enumerate(self.columns)\n }\n current_type = self._column_types\n result_type = current_type.assign(incoming_row_dict)\n if isinstance(result_type, _dtypes.InvalidType):\n raise TypeError(\n \"Data row contained incompatible types:\\n{}\".format(\n current_type.explain(incoming_row_dict)\n )\n )\n return result_type\n\n def get_dataframe(self):\n \"\"\"Returns a `pandas.DataFrame` of the table.\"\"\"\n pd = util.get_module(\n \"pandas\",\n required=\"Converting to pandas.DataFrame requires installing pandas\",\n )\n return pd.DataFrame.from_records(self.data, columns=self.columns)", "matching_document_document_id": "55d4d5f6-04e4-4090-bb7c-fb297871d8bc", "query_weight": 1.2541935483870967 }, { "query_id": "1faec26e-c651-4374-803e-e363f685475c", "query": "Weights & Biases documentation", "matching_document": "--- \n\n## description: Generated documentation for Weights & Biases APIs \n\n# Reference \n\nThese docs are automatically generated from the `wandb` library. \n\n### Reference sections \n\n1. Python Library: Add `wandb` to your script to capture metrics and save artifacts\n2. Command Line Interface: Log in, run jobs, execute sweeps, and more using shell commands\n3. JavaScript Library: A beta JavaScript/TypeScript client to track metrics from your Node server\n4. Weave: A beta query language to select and aggregate data \n\n### Examples and guides \n\nOur examples repo has scripts and colabs to try W&B features, and see integrations with various libraries. \n\nOur developer guide has guides, tutorials, and FAQs for the various W&B products.", "matching_document_document_id": "5276d09c-62fc-4d20-a129-290f142cc2e1", "query_weight": 0.9754838709677419 }, { "query_id": "03348bf3-9189-41b9-b29d-92da216a4c32", "query": "Methods to find W&B artifact storage location", "matching_document": "## Storage\n\nArtifact Storage \n\nW&B stores artifact files in a private Google Cloud Storage bucket located in the United States by default. All files are encrypted at rest and in transit. \n\nFor sensitive files, we recommend you set up Private Hosting or use reference artifacts. \n\nDuring training, W&B locally saves logs, artifacts, and configuration files in the following local directories: \n\n| File | Default location | To change default location set: |\n| --- | --- | --- |\n| logs | `./wandb` | `dir` in `wandb.init` or set the `WANDB_DIR` environment variable |\n| artifacts | `~/.cache/wandb` | the `WANDB_CACHE_DIR` environment variable |\n| configs | `~/.config/wandb` | the `WANDB_CONFIG_DIR` environment variable | \n\n:::caution\nDepending on the machine on `wandb` is initialized on, these default folders may not be located in a writeable part of the file system. This might trigger an error.\n::: \n\n### Clean up local artifact cache \n\n### How much storage does each artifact version use?", "matching_document_document_id": "2ec0a152-2b53-4183-8398-b61730217aac", "query_weight": 1.3935483870967742 }, { "query_id": "eb43d036-867d-4cd8-8333-5c7fb6997898", "query": "Bayesian optimization sweeps", "matching_document": "## Using W&B Sweeps with XGBoost\n### 1. Define the Sweep\nLet's walk through some of them together:\n\\* **Metric** \u2013 This is the metric the sweeps are attempting to optimize. Metrics can take a `name` (this metric should be logged by your training script) and a `goal` (`maximize` or `minimize`).\n\\* **Search Strategy** \u2013 Specified using the `\"method\"` key. We support several different search strategies with sweeps.\n\\* **Grid Search** \u2013 Iterates over every combination of hyperparameter values.\n\\* **Random Search** \u2013 Iterates over randomly chosen combinations of hyperparameter values.\n\\* **Bayesian Search** \u2013 Creates a probabilistic model that maps hyperparameters to probability of a metric score, and chooses parameters with high probability of improving the metric. The objective of Bayesian optimization is to spend more time in picking the hyperparameter values, but in doing so trying out fewer hyperparameter values.\n\\* **Parameters** \u2013 A dictionary containing the hyperparameter names, and discrete values, a range, or distributions from which to pull their values on each iteration.", "matching_document_document_id": "745ba1bc-4d49-4fd8-9ef5-ebdff7466343", "query_weight": 0.7432258064516128 }, { "query_id": "f0af7f3f-3770-481e-ad25-ec23946e0585", "query": "W&B enterprise features", "matching_document": "## Teams\n### Team Roles and Permissions\n* **Admin**: Team admins can add and remove other admins or team members. They have permissions to modify all projects and full deletion permissions. This includes, but is not limited to, deleting runs, projects, artifacts, and sweeps.\n* **Member**: A regular member of the team. A team member is invited by email by the team admin. A team member cannot invite other members. Team members can only delete runs and sweep runs created by that member. Suppose you have two members A and B. Member B moves a Run from team B's project to a different project owned by Member A. Member A can not delete the Run Member B moved to Member A's project. Only the member that creates the Run, or the team admin, can delete the run.\n* **Service (Enterprise-only feature)**: A service worker, an API key useful for using W&B with your run automation tools. If you use the API key from a service account for your team, make sure to set the environment variable **WANDB\\_USERNAME** to attribute runs to the correct user. See more on the relevant behavior below.\n* **View-Only (Enterprise-only feature)**: View-Only members can view assets within the team such as runs, reports, and workspaces. They can follow and comment on reports, but they can not create, edit, or delete project overview, reports, or runs. View-Only members do not have an API key.\n* **Custom roles (Enterprise-only feature)**: Custom roles allow organization admins to compose new roles by inheriting from the above **View-Only** or **Member** roles, and adding additional permissions to achieve fine-grained access control. Team admins can then assign any of those custom roles to users in their respective teams. Refer to this article for details.", "matching_document_document_id": "d04c7473-013b-4afa-8c71-ecb44bc24666", "query_weight": 1.3470967741935482 }, { "query_id": "f839d6d6-769d-4fcb-91ca-c2abc1dba2ee", "query": "Is it possible to change the default artifact cache location in Weights & Biases?", "matching_document": "## Storage\n\nArtifact Storage \n\nW&B stores artifact files in a private Google Cloud Storage bucket located in the United States by default. All files are encrypted at rest and in transit. \n\nFor sensitive files, we recommend you set up Private Hosting or use reference artifacts. \n\nDuring training, W&B locally saves logs, artifacts, and configuration files in the following local directories: \n\n| File | Default location | To change default location set: |\n| --- | --- | --- |\n| logs | `./wandb` | `dir` in `wandb.init` or set the `WANDB_DIR` environment variable |\n| artifacts | `~/.cache/wandb` | the `WANDB_CACHE_DIR` environment variable |\n| configs | `~/.config/wandb` | the `WANDB_CONFIG_DIR` environment variable | \n\n:::caution\nDepending on the machine on `wandb` is initialized on, these default folders may not be located in a writeable part of the file system. This might trigger an error.\n::: \n\n### Clean up local artifact cache \n\n### How much storage does each artifact version use?", "matching_document_document_id": "2ec0a152-2b53-4183-8398-b61730217aac", "query_weight": 0.1 }, { "query_id": "7c2ebd34-a233-4d61-8ce1-5dc6bbfea7d8", "query": "sweep configuration guide", "matching_document": "## Sweep configuration structure\n### Basic structure\ntrain.py\n```\nsweep\\_configuration = {\n\"name\": \"sweepdemo\",\n\"method\": \"bayes\",\n\"metric\": {\"goal\": \"minimize\", \"name\": \"validation\\_loss\"},\n\"parameters\": {\n\"learning\\_rate\": {\"min\": 0.0001, \"max\": 0.1},\n\"batch\\_size\": {\"values\": [16, 32, 64]},\n\"epochs\": {\"values\": [5, 10, 15]},\n\"optimizer\": {\"values\": [\"adam\", \"sgd\"]},\n},\n}\n\n``` \n\nDefine a sweep configuration in a YAML file if you want to manage sweeps interactively from the command line (CLI) \n\nconfig.yaml\n```\nprogram: train.py\nname: sweepdemo\nmethod: bayes\nmetric:\ngoal: minimize\nname: validation\\_loss\nparameters:\nlearning\\_rate:\nmin: 0.0001\nmax: 0.1\nbatch\\_size:\nvalues: [16, 32, 64]\nepochs:\nvalues: [5, 10, 15]\noptimizer:\nvalues: [\"adam\", \"sgd\"]\n\n```", "matching_document_document_id": "2a1ce862-7280-49c1-80c4-e4dfdd16914e", "query_weight": 0.7896774193548387 }, { "query_id": "d25c0e24-c064-49ad-b40d-a0b0c20242ef", "query": "What are the steps to properly delete and update artifacts in W&B?", "matching_document": "## Delete artifacts\n:::note\nArtifacts that are scheduled for deletion with a TTL policy, deleted with the W&B SDK, or deleted with the W&B App UI are first soft-deleted. Artifacts that are soft deleted undergo garbage collection before they are hard-deleted.\n::: \n\n### Delete an artifact version \n\nTo delete an artifact version: \n\n1. Select the name of the artifact. This will expand the artifact view and list all the artifact versions associated with that artifact.\n2. From the list of artifacts, select the artifact version you want to delete.\n3. On the right hand side of the workspace, select the kebab dropdown.\n4. Choose Delete. \n\nAn artifact version can also be deleted programatically via the delete() method. See the examples below. \n\n### Delete multiple artifact versions with aliases \n\nThe following code example demonstrates how to delete artifacts that have aliases associated with them. Provide the entity, project name, and run ID that created the artifacts. \n\n### Delete multiple artifact versions with a specific alias", "matching_document_document_id": "f4d54671-aed9-4d97-96af-8780b8507acd", "query_weight": 1.3935483870967742 }, { "query_id": "1292a074-98b6-4dae-8382-2578312d6ca1", "query": "How to use wandbtracker to log metadata in Weights & Biases?", "matching_document": "## Hugging Face Accelerate\n\nAccelerate is a library that enables the same PyTorch code to be run across any distributed configuration by adding just four lines of code, making training and inference at scale made simple, efficient and adaptable. \n\nAccelerate includes a Weights & Biases Tracker which we show how to use below. You can also read more about Accelerate Trackers in **their docs here** \n\n## Start logging with Accelerate \n\nTo get started with Accelerate and Weights & Biases you can follow the pseudocode below: \n\n## Accessing Accelerates' Internal W&B Tracker \n\nYou can quickly access the wandb tracker using the Accelerator.get\\_tracker() method. Just pass in the string corresponding to a tracker\u2019s .name attribute and it will return that tracker on the main process. \n\n```\nwandb\\_tracker = accelerator.get\\_tracker(\"wandb\")\n\n``` \n\nFrom there you can interact with wandb\u2019s run object like normal: \n\n```\nwandb\\_tracker.log\\_artifact(some\\_artifact\\_to\\_log)\n\n``` \n\n## Accelerate Articles", "matching_document_document_id": "4f08b4a7-bc90-4cf4-b3e4-d6224fde90fd", "query_weight": 0.5574193548387096 }, { "query_id": "92eeb2a7-53f0-4809-9d1b-87e62601556e", "query": "Integrating WandTracer with Weights & Biases projects", "matching_document": "## Applying Ethical Principles & Scoring to Langchain & Visualizing with W&B Prompts\n### WandbTracer: A Powerful Tool for Auto-Logging\nThe WandbTracer allows users to add a single line of code at the beginning of their script, which then captures the execution of LLM calls and tools in a visually appealing Trace panel in a Weights & Biases workspace. This tool not only logs the calls but also presents a table of information about each execution. Each row in the table represents one full execution of an independent LLM, chain, or agent. When clicked, it shows the trace for that execution, with the nested calls to our LLMs or any tooling that composes the execution. You can click on any part of the trace to see metadata related to that operation. \n\nIn practice, it looks like this: \n\n> \n\nTo set up the WandbTracer, you'll need to install the required packages and import the necessary libraries. Here's the code snippet to do that: \n\n```\n!pip -q wandb langchain\n...\nfrom wandb.integration.langchain import WandbTracer\nWandbTracer.init({\"project\": \"ethical-ada-llm-comparison\"})\n``` \n\n### What WandbTracer Lets You Do:", "matching_document_document_id": "67818c90-fd73-4a52-8a0c-ccf4a991497e", "query_weight": 0.7432258064516128 }, { "query_id": "49122049-f1f2-4614-be87-0e5429f5c824", "query": "Linking Weights & Biases run to model registry", "matching_document": "## Model registry\n### How it works\n1. **Log a model version**: In your training script, add a few lines of code to save the model files as an artifact to W&B.\n2. **Compare performance**: Check live charts to compare the metrics and sample predictions from model training and validation. Identify which model version performed the best.\n3. **Link to registry**: Bookmark the best model version by linking it to a registered model, either programmatically in Python or interactively in the W&B UI. \n\nThe following code snippet demonstrates how to log and link a model to the Model Registry: \n\n```python showLineNumbers\nimport wandb\nimport random\n\n# Start a new W&B run\n\nrun = wandb.init(project=\"models\\_quickstart\")\n\n# Simulate logging model metrics\n\nrun.log({\"acc\": random.random()})\n\n# Create a simulated model file\n\nwith open(\"my\\_model.h5\", \"w\") as f:\nf.write(\"Model: \" + str(random.random()))\n\n# Log and link the model to the Model Registry\n\nrun.link\\_model(path=\"./my\\_model.h5\", registered\\_model\\_name=\"MNIST\")\n\nrun.finish()\n```", "matching_document_document_id": "c8056eb1-14f6-47e4-82a9-b839839eb916", "query_weight": 1.0219354838709678 }, { "query_id": "64d24b02-21c9-41a7-81f4-f62861142e9b", "query": "accelerate launch arguments", "matching_document": "## Launching Distributed Code\nHuggingFace Accelerate has a special CLI command to help you launch your code in your system through accelerate launch. This command wraps around all of the different commands needed to launch your script on various platforms without you having to remember what each of them is. \n\nYou can launch your script quickly by using: \n\n```\naccelerate launch {script_name.py} --arg1 --arg2 ...\n``` \n\nJust put accelerate launch at the start of your command, and pass in additional arguments and parameters to your script afterward like normal! \n\nSince this runs the various torch spawn methods, all of the expected environment variables can be modified here as well. For example, here is how to use accelerate launch with a single GPU: \n\n```\nCUDA_VISIBLE_DEVICES=\"0\" accelerate launch {script_name.py} --arg1 --arg2 ...\n``` \n\nTo explore more options, you can take a look at the documentation [here](https://huggingface.co/docs/accelerate/v0.13.2/en/basic_tutorials/launch).", "matching_document_document_id": "98ab6d0b-2a8a-42d4-9393-4fe0cec531d9", "query_weight": 0.6503225806451612 }, { "query_id": "9082c9b9-be6e-4ab4-805a-61f7c21f0147", "query": "LangChain Weights & Biases compatibility", "matching_document": "## Prompts Quickstart\n### Use W&B Trace with LangChain\n#### 1. Set the LANGCHAIN\\_WANDB\\_TRACING environment variable\nFirst, set the LANGCHAIN\\_WANDB\\_TRACING environment variable to true. This will turn on automated Weights & Biases logging with LangChain: \n\n```\nimport os\n\n# turn on wandb logging for langchain\nos.environ[\"LANGCHAIN\\_WANDB\\_TRACING\"] = \"true\"\n\n``` \n\nThats it! Now any call to a LangChain LLM, Chain, Tool or Agent will be logged to Weights & Biases.", "matching_document_document_id": "6adad621-230b-42f0-badc-1738fef555a6", "query_weight": 1.2541935483870967 }, { "query_id": "bde1701f-5110-4d9d-a958-4e2abe7c3ca1", "query": "Best practices for integrating Weights & Biases in Python code", "matching_document": "## Add W&B to your code\n#### Training script with W&B Python SDK\n```python showLineNumbers\nimport wandb\nimport numpy as np\nimport random\n\n# Define sweep config\nsweep_configuration = {\n\"method\": \"random\",\n\"name\": \"sweep\",\n\"metric\": {\"goal\": \"maximize\", \"name\": \"val_acc\"},\n\"parameters\": {\n\"batch_size\": {\"values\": [16, 32, 64]},\n\"epochs\": {\"values\": [5, 10, 15]},\n\"lr\": {\"max\": 0.1, \"min\": 0.0001},\n},\n}\n\n# Initialize sweep by passing in config.\n# (Optional) Provide a name of the project.\nsweep_id = wandb.sweep(sweep=sweep_configuration, project=\"my-first-sweep\")\n\n# Define training function that takes in hyperparameter\n# values from `wandb.config` and uses them to train a\n# model and return metric\ndef train_one_epoch(epoch, lr, bs):\nacc = 0.25 + ((epoch / 30) + (random.random() / 10))\nloss = 0.2 + (1 - ((epoch - 1) / 10 + random.random() / 5))\nreturn acc, loss\n\ndef evaluate_one_epoch(epoch):\nacc = 0.1 + ((epoch / 20) + (random.random() / 10))\nloss = 0.25 + (1 - ((epoch - 1) / 10 + random.random() / 6))\nreturn acc, loss\n\ndef main():\nrun = wandb.init()\n\n# note that we define values from `wandb.config`", "matching_document_document_id": "34a34030-0144-44ab-9635-b3ae4894cc41", "query_weight": 1.2541935483870967 }, { "query_id": "fe0ba053-4da5-43d9-af16-b693f97bfa45", "query": "What Weights & Biases features support LLM application development?", "matching_document": "**Weights & Biases Prompts** is a suite of LLMOps tools built for the development of LLM-powered applications. \n\nUse W&B Prompts to visualize and inspect the execution flow of your LLMs, analyze the inputs and outputs of your LLMs, view the intermediate results and securely store and manage your prompts and LLM chain configurations. \n\n#### \ud83e\ude84 View Prompts In Action \n\n**In this notebook we will demostrate W&B Prompts:** \n\n* Using our 1-line LangChain integration\n* Using our Trace class when building your own LLM Pipelines \n\nSee here for the full W&B Prompts documentation \n\n## Installation \n\n```\n!pip install \"wandb>=0.15.4\" -qqq\n!pip install \"langchain>=0.0.218\" openai -qqq\n\n``` \n\n```\nimport langchain\nassert langchain.__version__ >= \"0.0.218\", \"Please ensure you are using LangChain v0.0.188 or higher\"\n\n``` \n\n## Setup \n\nThis demo requires that you have an OpenAI key \n\n# W&B Prompts \n\nW&B Prompts consists of three main components: \n\n**Trace table**: Overview of the inputs and outputs of a chain.", "matching_document_document_id": "5ccb89d6-6353-4b96-9920-060f32aad458", "query_weight": 1.207741935483871 }, { "query_id": "5b1e017c-c817-48a3-b3b0-3ee7ae37915f", "query": "What are the steps to manually stop a run in the Weights & Biases web interface?", "matching_document": "## Project Page\n### Workspace Tab\n* **Dot menu**: hover over a row in the sidebar to see the menu appear on the left side. Use this menu to rename a run, delete a run, or stop and active run.\n* **Visibility icon**: click the eye to turn on and off runs on graphs\n* **Color**: change the run color to another one of our presets or a custom color\n* **Search**: search runs by name. This also filters visible runs in the plots.\n* **Filter**: use the sidebar filter to narrow down the set of runs visible\n* **Group**: select a config column to dynamically group your runs, for example by architecture. Grouping makes plots show up with a line along the mean value, and a shaded region for the variance of points on the graph.\n* **Sort**: pick a value to sort your runs by, for example runs with the lowest loss or highest accuracy. Sorting will affect which runs show up on the graphs.\n* **Expand button**: expand the sidebar into the full table\n* **Run count**: the number in parentheses at the top is the total number of runs in the project. The number (N visualized) is the number of runs that have the eye turned on and are available to be visualized in each plot. In the example below, the graphs are only showing the first 10 of 183 runs. Edit a graph to increase the max number of runs visible.", "matching_document_document_id": "67a42e94-7673-43b4-bec6-8108db50f8a1", "query_weight": 0.1 }, { "query_id": "2de234bb-96d6-4652-9815-307cf4e1c32d", "query": "How to identify the best run in a Weights & Biases project?", "matching_document": "## Sweep\n\nView source on GitHub \n\nA set of runs associated with a sweep. \n\n```\nSweep(\nclient, entity, project, sweep\\_id, attrs=None\n)\n\n``` \n\n#### Examples: \n\nInstantiate with: \n\n```\napi = wandb.Api()\nsweep = api.sweep(path/to/sweep)\n\n``` \n\n| Attributes | |\n| --- | --- |\n| `runs` | (`Runs`) list of runs |\n| `id` | (str) sweep id |\n| `project` | (str) name of project |\n| `config` | (str) dictionary of sweep configuration |\n| `state` | (str) the state of the sweep |\n| `expected_run_count` | (int) number of expected runs for the sweep | \n\n## Methods \n\n### `best_run` \n\nView source \n\n```\nbest\\_run(\norder=None\n)\n\n``` \n\nReturn the best run sorted by the metric defined in config or the order passed in. \n\n### `display` \n\nView source \n\n```\ndisplay(\nheight=420, hidden=(False)\n) -> bool\n\n``` \n\nDisplay this object in jupyter. \n\n### `get` \n\nView source \n\n```\n@classmethod\nget(\nclient, entity=None, project=None, sid=None, order=None, query=None, \\*\\*kwargs\n)\n\n``` \n\n### `load` \n\nView source", "matching_document_document_id": "22d1890a-6b8a-47e3-93e5-47ad68eb9c99", "query_weight": 1.0219354838709678 }, { "query_id": "9692981d-295d-40de-a989-fe518020e2b4", "query": "wandb log_code", "matching_document": "## Run\n### Methods\n#### `log_code`\n| Arguments | |\n| --- | --- |\n| `root` | The relative (to `os.getcwd()`) or absolute path to recursively find code from. |\n| `name` | (str, optional) The name of our code artifact. By default, we'll name the artifact `source-$PROJECT_ID-$ENTRYPOINT_RELPATH`. There may be scenarios where you want many runs to share the same artifact. Specifying name allows you to achieve that. |\n| `include_fn` | A callable that accepts a file path and (optionally) root path and returns True when it should be included and False otherwise. This defaults to: `lambda path, root: path.endswith(\".py\")` |\n| `exclude_fn` | A callable that accepts a file path and (optionally) root path and returns `True` when it should be excluded and `False` otherwise. This defaults to a function that excludes all files within `<root>/.wandb/` and `<root>/wandb/` directories. | \n\n#### Examples: \n\nBasic usage \n\n```\nrun.log\\_code()\n\n``` \n\nAdvanced usage", "matching_document_document_id": "1c1ab624-25b4-465c-ac9d-75d589ae2177", "query_weight": 0.1 }, { "query_id": "b5216099-135e-4b69-9bee-dfa48e305c70", "query": "How can W&B reports be shared with non-W&B users?", "matching_document": "## Anonymous Mode\n\nAre you publishing code that you want anyone to be able to run easily? Use Anonymous Mode to let someone run your code, see a W&B dashboard, and visualize results without needing to create a W&B account first. \n\nAllow results to be logged in Anonymous Mode with `wandb.init(`**`anonymous=\"allow\"`**`)` \n\n:::info\n**Publishing a paper?** Please cite W&B, and if you have questions about how to make your code accessible while using W&B, reach out to us at support@wandb.com.\n::: \n\n### How does someone without an account see results? \n\nIf someone runs your script and you have to set `anonymous=\"allow\"`: \n\n### What happens to users with existing accounts? \n\nIf you set `anonymous=\"allow\"` in your script, we will check to make sure there's not an existing account first, before creating an anonymous account. This means that if a W&B user finds your script and runs it, their results will be logged correctly to their account, just like a normal run. \n\n## Example usage", "matching_document_document_id": "b16c8971-a81d-4473-a2b7-81b03075d7c4", "query_weight": 0.6038709677419355 }, { "query_id": "0eac2f9f-ecbc-42c3-8f09-f4ea38581c15", "query": "wandb Table data type compatibility", "matching_document": "## Table\n\nView source on GitHub \n\nThe Table class used to display and analyze tabular data. \n\n```\nTable(\ncolumns=None, data=None, rows=None, dataframe=None, dtype=None, optional=(True),\nallow\\_mixed\\_types=(False)\n)\n\n``` \n\nUnlike traditional spreadsheets, Tables support numerous types of data:\nscalar values, strings, numpy arrays, and most subclasses of `wandb.data_types.Media`.\nThis means you can embed `Images`, `Video`, `Audio`, and other sorts of rich, annotated media\ndirectly in Tables, alongside other traditional scalar values. \n\nThis class is the primary class used to generate the Table Visualizer\nin the UI: . \n\nTables can be constructed with initial data using the `data` or\n`dataframe` parameters:", "matching_document_document_id": "ef293fdf-142d-4a6e-b668-514932593e8f", "query_weight": 0.1 }, { "query_id": "7043c3e2-d19f-48b7-b862-255092fa0f07", "query": "How to download a W&B table as a Pandas DataFrame?", "matching_document": "## Export Table Data\n\nLike all W&B Artifacts, Tables can be converted into pandas dataframes for easy data exporting. \n\n## Convert `table` to `artifact` \n\nFirst, you'll need to convert the table to an artifact. The easiest way to do this using `artifact.get(table, \"table_name\")`: \n\n```\n# Create and log a new table.\nwith wandb.init() as r:\nartifact = wandb.Artifact(\"my\\_dataset\", type=\"dataset\")\ntable = wandb.Table(\ncolumns=[\"a\", \"b\", \"c\"], data=[(i, i \\* 2, 2\\*\\*i) for i in range(10)]\n)\nartifact.add(table, \"my\\_table\")\nwandb.log\\_artifact(artifact)\n\n# Retrieve the created table using the artifact you created.\nwith wandb.init() as r:\nartifact = r.use\\_artifact(\"my\\_dataset:latest\")\ntable = artifact.get(\"my\\_table\")\n\n``` \n\n## Convert `artifact` to Dataframe \n\nThen, convert the table into a dataframe: \n\n```\n# Following from the last code example:\ndf = table.get\\_dataframe()\n\n``` \n\n## Export Data \n\nNow you can export using any method dataframe supports:", "matching_document_document_id": "742d370f-04d6-468b-8dfb-2b241cb96084", "query_weight": 1.2541935483870967 }, { "query_id": "9f36e717-76b7-452a-9680-dc8854f6d641", "query": "Are there examples of YAML or JSON sweep configurations available in Weights & Biases documentation?", "matching_document": "## Sweep configuration structure\n### Basic structure\ntrain.py\n```\nsweep\\_configuration = {\n\"name\": \"sweepdemo\",\n\"method\": \"bayes\",\n\"metric\": {\"goal\": \"minimize\", \"name\": \"validation\\_loss\"},\n\"parameters\": {\n\"learning\\_rate\": {\"min\": 0.0001, \"max\": 0.1},\n\"batch\\_size\": {\"values\": [16, 32, 64]},\n\"epochs\": {\"values\": [5, 10, 15]},\n\"optimizer\": {\"values\": [\"adam\", \"sgd\"]},\n},\n}\n\n``` \n\nDefine a sweep configuration in a YAML file if you want to manage sweeps interactively from the command line (CLI) \n\nconfig.yaml\n```\nprogram: train.py\nname: sweepdemo\nmethod: bayes\nmetric:\ngoal: minimize\nname: validation\\_loss\nparameters:\nlearning\\_rate:\nmin: 0.0001\nmax: 0.1\nbatch\\_size:\nvalues: [16, 32, 64]\nepochs:\nvalues: [5, 10, 15]\noptimizer:\nvalues: [\"adam\", \"sgd\"]\n\n```", "matching_document_document_id": "2a1ce862-7280-49c1-80c4-e4dfdd16914e", "query_weight": 0.1 }, { "query_id": "ca2c35db-1e05-4e13-b671-07f67f800319", "query": "Weights & Biases best practices", "matching_document": "## \u270d\ufe0f W&B Best Practices\n\n1. **Projects**: Log multiple runs to a project to compare them. `wandb.init(project=\"project-name\")`\n2. **Groups**: For multiple processes or cross validation folds, log each process as a runs and group them together. `wandb.init(group=\"experiment-1\")`\n3. **Tags**: Add tags to track your current baseline or production model.\n4. **Notes**: Type notes in the table to track the changes between runs.\n5. **Reports**: Take quick notes on progress to share with colleagues and make dashboards and snapshots of your ML projects.", "matching_document_document_id": "aa2dbfa2-471e-4ec7-b1a2-60a608aa6873", "query_weight": 0.9754838709677419 }, { "query_id": "df14e37b-cb4c-4fd7-8f70-c4a0d3c7b597", "query": "using wandb for file tracking", "matching_document": "## Tracking Models and Datasets\n\nWandbCallback uses [artifacts](https://docs.wandb.com/artifacts/api) to keep track of models and datasets. \n\nTo track a model: \n\n* use `log_model=True` and `SaveModelCallback`\n* for custom scenarios, use function `log_model(path, name=None, metadata={})` \n\nTo track datasets: \n\n* use `log_dataset=True` to track the folder defined by `learn.dls.path`\n* use `log_dataset=\"my_path\"` to explicitly define a folder to track\n* optionally use `dataset_name` if you want a custom name, other,wise it is set to the folder name\n* for custom scenario, use function `log_dataset(path, name=None, metadata={})`\n* Note: the subfolder \"models\" is always ignored \n\nFinally, you can pull data from all these runs to create awesome interactive reports like this one, where your results are fully traceable! \n\nRefer to the [W&B documentation](https://docs.wandb.com/library/integrations/fastai) for more details. \n\n'", "matching_document_document_id": "732eac3d-f410-4823-98d2-894a82de2510", "query_weight": 0.1 }, { "query_id": "a67af25c-6316-4124-91f4-b8f09b981d0a", "query": "wandb distributed training setup", "matching_document": "## Experiments FAQ\n#### How can I use wandb with multiprocessing, e.g. distributed training?\n\nIf your training program uses multiple processes you will need to structure your program to avoid making wandb method calls from processes where you did not run `wandb.init()`.\\\n\\\nThere are several approaches to managing multiprocess training: \n\n1. Call `wandb.init` in all your processes, using the group keyword argument to define a shared group. Each process will have its own wandb run and the UI will group the training processes together.\n2. Call `wandb.init` from just one process and pass data to be logged over multiprocessing queues. \n\n:::info\nCheck out the Distributed Training Guide for more detail on these two approaches, including code examples with Torch DDP.\n:::", "matching_document_document_id": "cc461aea-36f0-4a9b-be45-e7110b8cfd88", "query_weight": 0.7432258064516128 }, { "query_id": "6eee9393-7866-4370-9141-b00deefcd429", "query": "wandb compatibility with LangChainAI", "matching_document": "## Prompts Quickstart\n### Use W&B Trace with LangChain\n#### 1. Set the LANGCHAIN\\_WANDB\\_TRACING environment variable\nFirst, set the LANGCHAIN\\_WANDB\\_TRACING environment variable to true. This will turn on automated Weights & Biases logging with LangChain: \n\n```\nimport os\n\n# turn on wandb logging for langchain\nos.environ[\"LANGCHAIN\\_WANDB\\_TRACING\"] = \"true\"\n\n``` \n\nThats it! Now any call to a LangChain LLM, Chain, Tool or Agent will be logged to Weights & Biases.", "matching_document_document_id": "6adad621-230b-42f0-badc-1738fef555a6", "query_weight": 1.2541935483870967 }, { "query_id": "f024e29b-daed-439f-abbb-a8e5194e68f9", "query": "What is the method to run sweeps on multiple GPUs in parallel?", "matching_document": "## Parallelize agents\n#### Parallelize on a multi-GPU machine\n1. Open more than one terminal window on your local machine.\n2. Specify the GPU instance to use with `CUDA_VISIBLE_DEVICES` when you start a W&B Sweep job (`wandb agent`). Assign `CUDA_VISIBLE_DEVICES` an integer value corresponding to the GPU instance to use. \n\nFor example, suppose you have two NVIDIA GPUs on your local machine. Open a terminal window and set `CUDA_VISIBLE_DEVICES` to `0` (`CUDA_VISIBLE_DEVICES=0`). Replace `sweep_ID` in the proceeding example with the W&B Sweep ID that is returned when you initialized a W&B Sweep: \n\nTerminal 1 \n\n```\nCUDA\\_VISIBLE\\_DEVICES=0 wandb agent sweep_ID\n\n``` \n\nOpen a second terminal window. Set `CUDA_VISIBLE_DEVICES` to `1` (`CUDA_VISIBLE_DEVICES=1`). Paste the same W&B Sweep ID for the `sweep_ID` mentioned in the proceeding code snippet: \n\nTerminal 2 \n\n```\nCUDA\\_VISIBLE\\_DEVICES=1 wandb agent sweep_ID\n\n```", "matching_document_document_id": "03228a82-a0d9-405b-ba85-e645187827e1", "query_weight": 1.207741935483871 }, { "query_id": "c573ef7a-b50e-4fe5-a50c-656441b2b466", "query": "How does wandb.save function for file management?", "matching_document": "## Save & Restore Files\n### Saving Files\n#### Save Policies and relative paths\n`wandb.save` accepts a **policy** argument which is set to \"**live**\" by default. Available policies are: \n\n* **live (default)** - sync this file to a wandb server immediately and re-sync it if it changes\n* **now** - sync this file to a wandb server immediately, don't continue syncing if it changes\n* **end** - only sync the file when the run finishes \n\nYou can also specify the **base\\_path** argument to `wandb.save`. This would allow you to maintain a directory hierarchy, for example: \n\n```\nwandb.save(path=\"./results/eval/\\*\", base\\_path=\"./results\", policy=\"now\")\n\n``` \n\nWould result in all files matching the pattern being saved in an `eval` folder instead of at the root.", "matching_document_document_id": "5b96d781-f99b-49d0-b3f3-36d44937441e", "query_weight": 0.7432258064516128 }, { "query_id": "372a6b0d-7c53-454d-a47b-7db01af51608", "query": "WandB features overview", "matching_document": "## Weights & Biases overview\nW&B is a platform that helps data scientists track their models, datasets, system information and more. With a few lines of code, you can start tracking everything about these features. It's free for personal use. Team use is normally a paid utility, but teams for academic purposes are free. You can use W&B with your favourite framework, like TensorFlow, Keras, PyTorch, Sklearn, fastai and many others. \n\nAll tracking information is sent to a dedicated project page on the W&B UI, where you can open high quality visualizations, aggregate information and compare models or parameters. One of the advantages of remotely storing the experiment\u2019s information is that it is easy to collaborate on the same project and share the results with your teammates. \n\nW&B provides 4 useful tools: \n\n* Dashboard: Experiment tracking\n* Artifacts: Dataset versioning, model versioning\n* Sweeps: Hyperparameter optimization\n* Reports: Save and share reproducible findings", "matching_document_document_id": "8c1600ad-9b6e-45cd-86ae-03afd8f80332", "query_weight": 0.9290322580645161 }, { "query_id": "db08ac6b-846c-4c6f-87d8-b0d9bb96acdc", "query": "Weights & Biases artifact access", "matching_document": "## Weave\n### Access Artifacts\n\nArtifacts are a core concept in W&B. They are a versioned, named collection of files and directories. Use Artifacts to track model weights, datasets, and any other file or directory. Artifacts are stored in W&B and can be downloaded or used in other runs. You can find more details and examples in this section of the report. Artifacts are normally accessed from the `project` object:\n\\* `project.artifactVersion()`: returns the specific artifact version for a given name and version within a project\n\\* `project.artifact(\"\")`: returns the artifact for a given name within a project. You can then use `.versions` to get a list of all versions of this artifact\n\\* `project.artifactType()`: returns the `artifactType` for a given name within a project. You can then use `.artifacts` to get a list of all artifacts with this type\n\\* `project.artifactTypes`: returns a list of all artifact types under the project", "matching_document_document_id": "7762795b-d90c-4e57-82cd-10b8f4426bff", "query_weight": 0.5109677419354839 }, { "query_id": "a4e2e3fa-f4c1-49a8-950b-6ce55cfdc2a4", "query": "In what scenarios does 'auto' resume a previous run or start a new one?", "matching_document": "metadata will be stored. When you call `download()` on an artifact,\n this is the directory where downloaded files will be saved. By default,\n this is the `./wandb` directory.\n resume: (bool, str, optional) Sets the resuming behavior. Options:\n `\"allow\"`, `\"must\"`, `\"never\"`, `\"auto\"` or `None`. Defaults to `None`.\n Cases:\n - `None` (default): If the new run has the same ID as a previous run,\n this run overwrites that data.\n - `\"auto\"` (or `True`): if the previous run on this machine crashed,\n automatically resume it. Otherwise, start a new run.\n - `\"allow\"`: if id is set with `init(id=\"UNIQUE_ID\")` or\n `WANDB_RUN_ID=\"UNIQUE_ID\"` and it is identical to a previous run,\n wandb will automatically resume the run with that id. Otherwise,\n wandb will start a new run.\n - `\"never\"`: if id is set with `init(id=\"UNIQUE_ID\")` or\n `WANDB_RUN_ID=\"UNIQUE_ID\"` and it is identical to a previous run,\n wandb will crash.", "matching_document_document_id": "bc7be183-f52f-4562-a551-f2113d440deb", "query_weight": 1.0219354838709678 }, { "query_id": "a6418ef6-e2ab-4e53-8db6-fe325491d946", "query": "How to use Weights & Biases groups for logging?", "matching_document": "## Weights & Biases Grouping\n```\n# Simulate launching multiple different jobs that log to the same experiment\n\nimport wandb\nimport math\nimport random\n\nfor i in range(5):\njob_type = \"rollout\"\nif i == 2:\njob_type = \"eval\"\nif i == 3:\njob_type = \"eval2\"\nif i == 4:\njob_type = \"optimizer\"\n\n# Set group and job_type to see auto-grouping in the UI\nwandb.init(project=\"group-demo\",\ngroup=\"exp_\" + str(experiment_index),\njob_type=job_type)\n\nfor j in range(100):\nacc = 0.1 * (math.log(1 + j + .1) + random.random())\nval_acc = 0.1 * (math.log(1+ j + 2) + random.random() + random.random())\nif (j % 10 == 0):\nwandb.log({\"acc\":acc, \"val_acc\":val_acc})\n\n# Using this to mark a run complete in a notebook context\nwandb.finish()\n\n# I'm incrementing this so you can re-run this cell and get another experiment\n# grouped in the W&B UI\nexperiment_index += 1\n\n```", "matching_document_document_id": "e271dae7-8483-477d-82cc-35ee13045749", "query_weight": 1.8116129032258064 }, { "query_id": "920b6b5a-dd75-4d41-bc07-df9b3903e3a4", "query": "wandb network connectivity errors", "matching_document": "## Troubleshooting\n#### How do I deal with network issues?\n\nIf you're seeing SSL or network errors:`wandb: Network error (ConnectionError), entering retry loop`. You can try a couple of different approaches to solving this issue: \n\n1. Upgrade your SSL certificate. If you're running the script on an Ubuntu server, run `update-ca-certificates` We can't sync training logs without a valid SSL certificate because it's a security vulnerability.\n2. If your network is flaky, run training in offline mode and sync the files to us from a machine that has Internet access.\n3. Try running W&B Private Hosting, which operates on your machine and doesn't sync files to our cloud servers. \n\n`SSL CERTIFICATE_VERIFY_FAILED`: this error could be due to your company's firewall. You can set up local CAs and then use: \n\n`export REQUESTS_CA_BUNDLE=/etc/ssl/certs/ca-certificates.crt`", "matching_document_document_id": "eba9b6dc-4582-4539-bf24-39b10a2e6960", "query_weight": 1.207741935483871 }, { "query_id": "da6525a1-ed37-4b6b-85ce-4f4bd66fd3ee", "query": "Best practices for naming and grouping runs in Weights & Biases", "matching_document": "## \u270d\ufe0f W&B Best Practices\n\n1. **Projects**: Log multiple runs to a project to compare them. `wandb.init(project=\"project-name\")`\n2. **Groups**: For multiple processes or cross validation folds, log each process as a runs and group them together. `wandb.init(group=\"experiment-1\")`\n3. **Tags**: Add tags to track your current baseline or production model.\n4. **Notes**: Type notes in the table to track the changes between runs.\n5. **Reports**: Take quick notes on progress to share with colleagues and make dashboards and snapshots of your ML projects.", "matching_document_document_id": "aa2dbfa2-471e-4ec7-b1a2-60a608aa6873", "query_weight": 1.3470967741935482 }, { "query_id": "87a59e12-82be-467d-9192-df8030cf6fe7", "query": "Can you track the model name automatically in W&B?", "matching_document": "## Track a model\n\nTrack a model, the model's dependencies, and other information relevant to that model with the W&B Python SDK. \n\nUnder the hood, W&B creates a lineage of model artifact that you can view with the W&B App UI or programmatically with the W&B Python SDK. See the Create model lineage map for more information. \n\n## How to log a model \n\nUse the `run.log_model` API to log a model. Provide the path where your model files are saved to the `path` parameter. The path can be a local file, directory, or reference URI to an external bucket such as `s3://bucket/path`. \n\nOptionally provide a name for the model artifact for the `name` parameter. If `name` is not specified, W&B uses the basename of the input path prepended with the run ID. \n\nCopy and paste the proceeding code snippet. Ensure to replace values enclosed in `<>` with your own.", "matching_document_document_id": "fdcd64f1-b177-42e2-88b2-6df060ec5b67", "query_weight": 0.1 }, { "query_id": "d4f4da6d-3973-4086-b1e9-798704f4aaeb", "query": "data privacy wandb", "matching_document": "## FAQ\nPlease see our [technical FAQ](https://wandb.ai/wandb/getting-started/reports/Technical-FAQ--VmlldzoyMTA5MDM) for commonly asked questions about the API and product. \n\n## Who Has Rights to the Data? \n\nYou can always export and delete your data at any time. We will never share data associated with private projects. We hope that when you can, you will make your work public so that other practitioners can learn from it. \n\nWe hope to discover and share high level patterns to move the field of machine learning forward. For example, we wrote [this article](https://www.wandb.com/articles/monitor-improve-gpu-usage-for-model-training) on how people are not fully utilizing their GPUs. We want to do this in a way that respects your privacy and feels honest. If you have any concerns about data privacy, we'd love to hear from you. Reach out at contact@wandb.com. \n\n## Why Are You Building These Tools?", "matching_document_document_id": "823744ee-3898-4bb2-a151-84312fcec1c8", "query_weight": 0.6967741935483871 }, { "query_id": "fae1a027-f1e0-4d70-a1a7-bb8a4ec40349", "query": "download artifacts", "matching_document": "## Download and use artifacts\n#### Download and use an artifact stored on W&B\nUse the object returned to download all the contents of the artifact: \n\n```\ndatadir = artifact.download()\n\n``` \n\nYou can optionally pass a path to the root parameter to download the contents of the artifact to a specific directory. For more information, see the Python SDK Reference Guide. \n\nUse the `get_path` method to download only subset of files: \n\n```\npath = artifact.get\\_path(name)\n\n``` \n\nThis fetches only the file at the path `name`. It returns an `Entry` object with the following methods: \n\n* `Entry.download`: Downloads file from the artifact at path `name`\n* `Entry.ref`: If the entry was stored as a reference using `add_reference`, returns the URI \n\nReferences that have schemes that W&B knows how to handle can be downloaded just like artifact files. For more information, see Track external files. \n\nFirst, import the W&B SDK. Next, create an artifact from the Public API Class. Provide the entity, project, artifact, and alias associated with that artifact:", "matching_document_document_id": "a42e4236-eca7-4763-a693-b44e8187ca6f", "query_weight": 1.904516129032258 }, { "query_id": "0ec8c423-7c42-4c26-a12f-7379a71be575", "query": "Is an API key mandatory for downloading artifacts from W&B?", "matching_document": "## Download and use artifacts\n#### Download and use an artifact stored on W&B\nUse the object returned to download all the contents of the artifact: \n\n```\ndatadir = artifact.download()\n\n``` \n\nYou can optionally pass a path to the root parameter to download the contents of the artifact to a specific directory. For more information, see the Python SDK Reference Guide. \n\nUse the `get_path` method to download only subset of files: \n\n```\npath = artifact.get\\_path(name)\n\n``` \n\nThis fetches only the file at the path `name`. It returns an `Entry` object with the following methods: \n\n* `Entry.download`: Downloads file from the artifact at path `name`\n* `Entry.ref`: If the entry was stored as a reference using `add_reference`, returns the URI \n\nReferences that have schemes that W&B knows how to handle can be downloaded just like artifact files. For more information, see Track external files. \n\nFirst, import the W&B SDK. Next, create an artifact from the Public API Class. Provide the entity, project, artifact, and alias associated with that artifact:", "matching_document_document_id": "a42e4236-eca7-4763-a693-b44e8187ca6f", "query_weight": 1.2541935483870967 }, { "query_id": "e3017f2c-3526-4f44-8f84-1d94d63e7373", "query": "Python script Wandb configuration", "matching_document": "## Add W&B to your code\n#### Training script with W&B Python SDK\n```python showLineNumbers\nimport wandb\nimport numpy as np\nimport random\n\n# Define sweep config\nsweep_configuration = {\n\"method\": \"random\",\n\"name\": \"sweep\",\n\"metric\": {\"goal\": \"maximize\", \"name\": \"val_acc\"},\n\"parameters\": {\n\"batch_size\": {\"values\": [16, 32, 64]},\n\"epochs\": {\"values\": [5, 10, 15]},\n\"lr\": {\"max\": 0.1, \"min\": 0.0001},\n},\n}\n\n# Initialize sweep by passing in config.\n# (Optional) Provide a name of the project.\nsweep_id = wandb.sweep(sweep=sweep_configuration, project=\"my-first-sweep\")\n\n# Define training function that takes in hyperparameter\n# values from `wandb.config` and uses them to train a\n# model and return metric\ndef train_one_epoch(epoch, lr, bs):\nacc = 0.25 + ((epoch / 30) + (random.random() / 10))\nloss = 0.2 + (1 - ((epoch - 1) / 10 + random.random() / 5))\nreturn acc, loss\n\ndef evaluate_one_epoch(epoch):\nacc = 0.1 + ((epoch / 20) + (random.random() / 10))\nloss = 0.25 + (1 - ((epoch - 1) / 10 + random.random() / 6))\nreturn acc, loss\n\ndef main():\nrun = wandb.init()\n\n# note that we define values from `wandb.config`", "matching_document_document_id": "34a34030-0144-44ab-9635-b3ae4894cc41", "query_weight": 0.1 }, { "query_id": "6cd26049-f3dc-41ba-8389-635f3c36569d", "query": "WandB API methods", "matching_document": "## Api\n\nView source on GitHub \n\nUsed for querying the wandb server. \n\n```\nApi(\noverrides=None,\ntimeout: Optional[int] = None,\napi\\_key: Optional[str] = None\n) -> None\n\n``` \n\n#### Examples: \n\nMost common way to initialize \n\n```\n>>> wandb.Api()\n\n``` \n\n| Arguments | |\n| --- | --- |\n| `overrides` | (dict) You can set `base_url` if you are using a wandb server other than . You can also set defaults for `entity`, `project`, and `run`. | \n\n| Attributes | |\n| --- | --- |\n| | | \n\n## Methods \n\n### `artifact` \n\nView source \n\n```\nartifact(\nname, type=None\n)\n\n``` \n\nReturn a single artifact by parsing path in the form `entity/project/name`. \n\n| Arguments | |\n| --- | --- |\n| `name` | (str) An artifact name. May be prefixed with entity/project. Valid names can be in the following forms: name:version name:alias |\n| `type` | (str, optional) The type of artifact to fetch. | \n\n| Returns | |\n| --- | --- |\n| A `Artifact` object. | | \n\n### `artifact_type` \n\nView source \n\n### `flush`", "matching_document_document_id": "8d4ccf38-8646-42b5-abfa-b46e0c4ca9f5", "query_weight": 0.1 }, { "query_id": "08338258-9c14-42de-91b6-c1f1b34a37dc", "query": "Where can I find documentation on defining a sweep plan for Weights & Biases?", "matching_document": "## Sweep configuration structure\n### Basic structure\ntrain.py\n```\nsweep\\_configuration = {\n\"name\": \"sweepdemo\",\n\"method\": \"bayes\",\n\"metric\": {\"goal\": \"minimize\", \"name\": \"validation\\_loss\"},\n\"parameters\": {\n\"learning\\_rate\": {\"min\": 0.0001, \"max\": 0.1},\n\"batch\\_size\": {\"values\": [16, 32, 64]},\n\"epochs\": {\"values\": [5, 10, 15]},\n\"optimizer\": {\"values\": [\"adam\", \"sgd\"]},\n},\n}\n\n``` \n\nDefine a sweep configuration in a YAML file if you want to manage sweeps interactively from the command line (CLI) \n\nconfig.yaml\n```\nprogram: train.py\nname: sweepdemo\nmethod: bayes\nmetric:\ngoal: minimize\nname: validation\\_loss\nparameters:\nlearning\\_rate:\nmin: 0.0001\nmax: 0.1\nbatch\\_size:\nvalues: [16, 32, 64]\nepochs:\nvalues: [5, 10, 15]\noptimizer:\nvalues: [\"adam\", \"sgd\"]\n\n```", "matching_document_document_id": "2a1ce862-7280-49c1-80c4-e4dfdd16914e", "query_weight": 0.5574193548387096 }, { "query_id": "1db28885-619f-410a-b41d-761af23f19fb", "query": "best practices for dataset versioning", "matching_document": "## Dataset Versioning\n### Visualize & easily share your data workflow\n* choose **meaningful organizational types** for your artifacts and jobs: for data, this could be `train`, `val`, or `test`; for scripts this could be `preprocess`, `train`, `evaluate`, etc. You may also want to log other data as artifacts: a model's predictions on fixed validation data, samples of generated output, evaluation metrics, etc.\n* **explore the artifact graph**: interact with all the connections between your code and data (input artifact(s) \u2192 script or job \u2192 output artifact(s)). Click \"explode\" on the compute graph to see all the versions for each artifact or all the runs of each script by job type. Click on individual nodes to see further details (file contents or code, annotations/metadata, config, timestamp, parent/child nodes, etc).\n* **iterate confidently**: all your experimental script runs and data will be saved and versioned automatically, so you can focus on the core modeling task and not worry about where and when you saved which version of your dataset or code\n* **share & replicate easily**: once you've integrated artifacts, you and your teammates can smoothly rerun the same workflow and pull from identical datasets (defaulting to the latest/best version), even to train in a different context/on different hardware", "matching_document_document_id": "f45869fa-ad3d-428e-86cd-94784639cb4f", "query_weight": 1.904516129032258 }, { "query_id": "6fd7beeb-ed4b-43b7-8265-c9663462d38c", "query": "Tutorial on integrating LangChain with Weights & Biases", "matching_document": "## Prompts Quickstart\n### Use W&B Trace with LangChain\n#### 1. Set the LANGCHAIN\\_WANDB\\_TRACING environment variable\nFirst, set the LANGCHAIN\\_WANDB\\_TRACING environment variable to true. This will turn on automated Weights & Biases logging with LangChain: \n\n```\nimport os\n\n# turn on wandb logging for langchain\nos.environ[\"LANGCHAIN\\_WANDB\\_TRACING\"] = \"true\"\n\n``` \n\nThats it! Now any call to a LangChain LLM, Chain, Tool or Agent will be logged to Weights & Biases.", "matching_document_document_id": "6adad621-230b-42f0-badc-1738fef555a6", "query_weight": 1.2541935483870967 }, { "query_id": "e71880a5-5efb-4f06-8068-88f9059c6970", "query": "Best practices for naming runs in W&B to avoid conflicts", "matching_document": "## \u270d\ufe0f W&B Best Practices\n\n1. **Projects**: Log multiple runs to a project to compare them. `wandb.init(project=\"project-name\")`\n2. **Groups**: For multiple processes or cross validation folds, log each process as a runs and group them together. `wandb.init(group=\"experiment-1\")`\n3. **Tags**: Add tags to track your current baseline or production model.\n4. **Notes**: Type notes in the table to track the changes between runs.\n5. **Reports**: Take quick notes on progress to share with colleagues and make dashboards and snapshots of your ML projects.", "matching_document_document_id": "aa2dbfa2-471e-4ec7-b1a2-60a608aa6873", "query_weight": 0.1 }, { "query_id": "6e9404ec-c32f-40eb-add3-255a2694cf53", "query": "Examples of markdown usage in Weights & Biases", "matching_document": "## Edit a report\n#### Markdown\nAdd markdown to your report interactively with the App UI or with the W&B SDK. \n\nEnter a forward slash (`/`) in the report to display a dropdown menu. From the dropdown choose **Markdown**. \n\nUse the `wandb.apis.reports.MarkdownBlock` Class to create a markdown block programmatically. Pass a string to the `text` parameter: \n\n```\nimport wandb\nimport wandb.apis.reports as wr\n\nreport = wr.Report(project=\"report-editing\")\n\nreport.blocks = [\nwr.MarkdownBlock(text=\"Markdown cell with \\*italics\\* and \\*\\*bold\\*\\* and $e=mc^2$\")\n]\n\n``` \n\nThis will render a markdown block similar to:", "matching_document_document_id": "c651c051-93c3-41b2-bfc8-844d8e467a7f", "query_weight": 0.1 }, { "query_id": "75110427-2c8e-4b1e-8f0e-526d1e4e2b06", "query": "Optimal metric logging strategies for Bayesian optimization sweeps", "matching_document": "## 4\ufe0f\u20e3 Define the Sweep\n#### \ud83d\udc48 Pick a `method`\nWe provide the following search `methods`:\n\\* **`grid` Search** \u2013 Iterate over every combination of hyperparameter values.\nVery effective, but can be computationally costly.\n\\* **`random` Search** \u2013 Select each new combination at random according to provided `distribution`s. Surprisingly effective!\n\\* **`bayes`ian Search** \u2013 Create a probabilistic model of metric score as a function of the hyperparameters, and choose parameters with high probability of improving the metric. Works well for small numbers of continuous parameters but scales poorly. \n\nWe'll stick with `random`. \n\n```\nsweep_config = {\n'method': 'bayes'\n}\n\n``` \n\nFor `bayes`ian Sweeps,\nyou also need to tell us a bit about your `metric`.\nWe need to know its `name`, so we can find it in the model outputs\nand we need to know whether your `goal` is to `minimize` it\n(e.g. if it's the squared error)\nor to `maximize` it\n(e.g. if it's the accuracy). \n\n```\nmetric = {\n'name': 'val_loss',\n'goal': 'minimize'\n}\n\nsweep_config['metric'] = metric\n\n```", "matching_document_document_id": "96a35a08-c7ed-44f6-9ed5-cc851c11dd67", "query_weight": 0.7432258064516128 }, { "query_id": "78f2c1fa-5774-4567-bbea-4e807e9230cb", "query": "Is there a recommended frequency for metric logging during Bayesian optimization sweeps?", "matching_document": "## Limits & Performance\n### Best Practices for Fast Pages\n#### Logged Metrics\n##### Metric Frequency\nPick a logging frequency that is appropriate to the metric you are logging. As a general rule of thumb, the wider the metric the less frequently you should log it. Concretely, we recommend: \n\n* **Scalars**: <100,000 logged points per metric\n* **Media**: <50,000 logged points per metric\n* **Histograms**: <10,000 logged points per metric \n\n:::caution \n\n::: \n\n```\n# Training loop with 1m total steps\nfor step in range(1000000):\n# \u274c not recommended\nwandb.log(\n{\n\"scalar\": step, # 100,000 scalars\n\"media\": wandb.Image(...), # 100,000 images\n\"histogram\": wandb.Histogram(...), # 100,000 histograms\n}\n)\n\n# \u2705 recommended\nif step % 1000 == 0:\nwandb.log(\n{\n\"histogram\": wandb.Histogram(...), # 10,000 histograms\n},\ncommit=False,\n)\nif step % 200 == 0:\nwandb.log(\n{\n\"media\": wandb.Image(...), # 50,000 images\n},\ncommit=False,\n)\nif step % 100 == 0:\nwandb.log(\n{\n\"scalar\": step, # 100,000 scalars\n},\ncommit=True,\n) # Commit batched, per-step metrics together\n\n```", "matching_document_document_id": "1f22afa6-1f66-4a8c-bdd4-735b81f25e93", "query_weight": 0.7432258064516128 }, { "query_id": "7e8225ff-28ee-44d3-9b7f-660486ee18b6", "query": "What are the best practices for logging metrics during Bayesian optimization sweeps?", "matching_document": "## instead of defining hard values\nCopy the sweep ID and replace `sweepID` in the proceeding code snippet to start the sweep job with the `wandb agent` command: \n\n```\nwandb agent --count $NUM your-entity/sweep-demo-cli/sweepID\n\n``` \n\nFor more information on how to start sweep jobs, see Start sweep jobs. \n\n## Consideration when logging metrics \n\nEnsure to log the metric you specify in your sweep configuration explicitly to W&B. Do not log metrics for your sweep inside of a sub-directory. \n\nFor example, consider the proceeding psuedocode. A user wants to log the validation loss (`\"val_loss\": loss`). First they pass the values into a dictionary (line 16). However, the dictionary passed to `wandb.log` does not explicitly access the key-value pair in the dictionary:", "matching_document_document_id": "529f4145-2639-44b3-9e84-285f9b520c49", "query_weight": 0.7432258064516128 }, { "query_id": "eaf4167f-7f0f-4f2a-ae96-410867ffb103", "query": "optimization metric tracking", "matching_document": "## GCP with Weights & Biases\n### Tracking metrics: Using W&B to keep track of model\u2019s performance\n```\n# Compile the model\nmodel.compile(loss=loss, optimizer=optimizer, metrics=['accuracy'])\n\n# The WandbCallback logs metrics and some examples of the test data\nmodel.fit(X_train, y_train, validation_data=(X_test, y_test),\nepochs=config.epochs,\ncallbacks=[WandbCallback(data_type=\"image\", labels=labels),\nTensorBoard(log_dir=wandb.run.dir)])\n``` \n\n* Two things to pay attention to in the above code block:\n* Note how the number of epochs has been specified. Instead of just supplying a hard-coded value, we are supplying config.epochs which we defined earlier. Doing this will allow W&B to keep track of the configuration variables as well.\n* WandbCallback(data_type=\"image\", labels=labels): We can simply supply WandbCallback without any arguments and it will keep track of the performance of the model. Additionally, if we supply the data_type and labels arguments, W&B will log some sample predictions on the validation data points as the model is training.- -", "matching_document_document_id": "72e0cc30-3dde-47c1-a683-21c8b15e07c9", "query_weight": 0.7432258064516128 }, { "query_id": "9cb8d8d9-5aa5-43fc-bc2f-8d9a07e75ebf", "query": "Using Weights & Biases for multimedia machine learning projects", "matching_document": "## Log Media & Objects\n\nWe support images, video, audio, and more. Log rich media to explore your results and visually compare your runs, models, and datasets. Read on for examples and how-to guides. \n\n:::info\nLooking for reference docs for our media types? You want this page.\n::: \n\n:::info\nYou can see working code to log all of these media objects in this Colab Notebook, check out what the results look like on wandb.ai here, and follow along with a video tutorial, linked above.\n::: \n\n## Images \n\nLog images to track inputs, outputs, filter weights, activations, and more! \n\nImages can be logged directly from NumPy arrays, as PIL images, or from the filesystem. \n\n:::info\nIt's recommended to log fewer than 50 images per step to prevent logging from becoming a bottleneck during training and image loading from becoming a bottleneck when viewing results.\n::: \n\nProvide arrays directly when constructing images manually, e.g. using `make_grid` from `torchvision`. \n\nArrays are converted to png using Pillow.", "matching_document_document_id": "167ebee2-6600-4d4b-935a-95282e1abe8f", "query_weight": 1.2541935483870967 }, { "query_id": "335f9447-2065-4939-8a2f-56508d1371cf", "query": "How to remove Weights & Biases hooks from PyTorch models?", "matching_document": "def unwatch(models=None):\n \"\"\"Remove pytorch model topology, gradient and parameter hooks.\n\n Args:\n models: (list) Optional list of pytorch models that have had watch called on them\n \"\"\"\n if models:\n if not isinstance(models, (tuple, list)):\n models = (models,)\n for model in models:\n if not hasattr(model, \"_wandb_hook_names\"):\n wandb.termwarn(\"%s model has not been watched\" % model)\n else:\n for name in model._wandb_hook_names:\n wandb.run._torch.unhook(name)\n delattr(model, \"_wandb_hook_names\")\n # TODO: we should also remove recursively model._wandb_watch_called\n\n else:\n wandb.run._torch.unhook_all()", "matching_document_document_id": "82e6e3a9-d9f0-4a47-b3ef-66fbe33dc419", "query_weight": 1.2541935483870967 }, { "query_id": "ab21df40-2f12-4cfd-b723-d9cca417d0a3", "query": "NLP large language model integration", "matching_document": "--- \n\n## displayed\\_sidebar: default \n\nimport { CTAButtons } from '@site/src/components/CTAButtons/CTAButtons.tsx'; \n\n# Hugging Face Transformers \n\nThe Hugging Face Transformers library makes state-of-the-art NLP models like BERT and training techniques like mixed precision and gradient checkpointing easy to use. The W&B integration adds rich, flexible experiment tracking and model versioning to interactive centralized dashboards without compromising that ease of use. \n\n## \ud83e\udd17 Next-level logging in few lines \n\n```\nos.environ[\"WANDB\\_PROJECT\"] = \"\" # name your W&B project\nos.environ[\"WANDB\\_LOG\\_MODEL\"] = \"checkpoint\" # log all model checkpoints\n\nfrom transformers import TrainingArguments, Trainer\n\nargs = TrainingArguments(..., report\\_to=\"wandb\") # turn on W&B logging\ntrainer = Trainer(..., args=args)\n\n``` \n\n:::info\nIf you'd rather dive straight into working code, check out this Google Colab.\n::: \n\n## Getting started: track experiments \n\n### 2) Name the project", "matching_document_document_id": "fe7a80f3-1c69-44cf-ab7c-fa24ac3e77dd", "query_weight": 0.9754838709677419 }, { "query_id": "60209be7-c521-4249-9ce0-7cab8b93226c", "query": "I want to create my own integration to properly track the outputs of a NLP large language model. How can I make it automatically log for me in a single line of code", "matching_document": "## Applying Ethical Principles & Scoring to Langchain & Visualizing with W&B Prompts\n### WandbTracer: A Powerful Tool for Auto-Logging\nThe WandbTracer allows users to add a single line of code at the beginning of their script, which then captures the execution of LLM calls and tools in a visually appealing Trace panel in a Weights & Biases workspace. This tool not only logs the calls but also presents a table of information about each execution. Each row in the table represents one full execution of an independent LLM, chain, or agent. When clicked, it shows the trace for that execution, with the nested calls to our LLMs or any tooling that composes the execution. You can click on any part of the trace to see metadata related to that operation. \n\nIn practice, it looks like this: \n\n> \n\nTo set up the WandbTracer, you'll need to install the required packages and import the necessary libraries. Here's the code snippet to do that: \n\n```\n!pip -q wandb langchain\n...\nfrom wandb.integration.langchain import WandbTracer\nWandbTracer.init({\"project\": \"ethical-ada-llm-comparison\"})\n``` \n\n### What WandbTracer Lets You Do:", "matching_document_document_id": "67818c90-fd73-4a52-8a0c-ccf4a991497e", "query_weight": 0.9754838709677419 }, { "query_id": "126b33ad-0c4b-4edc-8f6e-44564d5edef3", "query": "Who uses Weights & Biases", "matching_document": "## Better Models Faster with Weights & Biases\n* Track and compare models: I want to test out my hypotheses fast and iterate quickly to find the best model\n* Visualize model performance for debugging: I want to see how my models are performing in real time and debug them\n* Efficient hyperparameter search: I want to find the best model faster than everyone else\n* Resource efficient model training: I want to be efficient with my model training and not spend more money than I need to\n* Show off my model: I want to share my models and key insights with my teammates and the Kaggle community \n\nCarlo Lepelaars and Mani Sarkar, two of our beta users finished 5th place out of 2255 on the SoftBank Forex Algorithm Challenge and found [using Weights & Biases](https://github.com/neomatrix369/awesome-ai-ml-dl/blob/master/examples/cloud-devops-infra/wandb/sb-fx-competition/README.md#tracking-sb-fx-competition-experiments-with-weights--biases-) extremely helpful for tuning their model\u2019s performance.", "matching_document_document_id": "391f349f-63f1-4bf2-af6b-fdcd8b69101f", "query_weight": 0.7432258064516128 }, { "query_id": "1f0a9124-640d-41b5-96e9-85aacb1d3d22", "query": "Weights & Biases run naming", "matching_document": "## Run\n| Attributes | |\n| --- | --- |\n| `summary` | (Summary) Single values set for each `wandb.log()` key. By default, summary is set to the last value logged. You can manually set summary to the best value, like max accuracy, instead of the final value. |\n| `config` | Config object associated with this run. |\n| `dir` | The directory where files associated with the run are saved. |\n| `entity` | The name of the W&B entity associated with the run. Entity can be a username or the name of a team or organization. |\n| `group` | Name of the group associated with the run. Setting a group helps the W&B UI organize runs in a sensible way. If you are doing a distributed training you should give all of the runs in the training the same group. If you are doing cross-validation you should give all the cross-validation folds the same group. |\n| `id` | Identifier for this run. |\n| `mode` | For compatibility with `0.9.x` and earlier, deprecate eventually. |\n| `name` | Display name of the run. Display names are not guaranteed to be unique and may be descriptive. By default, they are randomly generated. |\n| `notes` | Notes associated with the run, if there are any. Notes can be a multiline string and can also use markdown and latex equations inside `$$`, like `$x + 3$`. |\n| `path` | Path to the run. Run paths include entity, project, and run ID, in the format `entity/project/run_id`. |\n| `project` | Name of the W&B project associated with the run. |\n| `resumed` | True if the run was resumed, False otherwise. |\n| `settings` | A frozen copy of run's Settings object. |\n| `start_time` | Unix timestamp (in seconds) of when the run started. |\n| `starting_step` | The first step of the run. |\n| `step` | Current value of the step. This counter is incremented by `wandb.log`. |\n| `sweep_id` | ID of the sweep associated with the run, if there is one. |", "matching_document_document_id": "cdf82567-6205-4b8c-a0e7-ee2333c5bcde", "query_weight": 0.9754838709677419 }, { "query_id": "2978c63f-cdc8-473c-9164-c9587f80fc71", "query": "Weights & Biases troubleshooting", "matching_document": "## Troubleshoot Sweeps\n\nTroubleshoot W&B Sweeps \n\nTroubleshoot common error messages with the guidance suggested. \n\n### `CommError, Run does not exist` and `ERROR Error uploading` \n\nYour W&B Run ID might be defined if these two error messages are both returned. As an example, you might have a similar code snippet defined somewhere in your Jupyter Notebooks or Python script: \n\n```\nwandb.init(id=\"some-string\")\n\n``` \n\nYou can not set a Run ID for W&B Sweeps because W&B automatically generates random, unique IDs for Runs created by W&B Sweeps. \n\nW&B Run IDs need to be unique within a project. \n\nWe recommend you pass a name to the name parameter when you initialized W&B, if you want to set a custom name that will appear on tables and graphs. For example: \n\n```\nwandb.init(name=\"a helpful readable run name\")\n\n``` \n\n### `Cuda out of memory` \n\n### `anaconda 400 error` \n\nThe following error usually occurs when you do not log the metric that you are optimizing:", "matching_document_document_id": "13152c97-59c9-456d-b955-55a590204565", "query_weight": 0.9754838709677419 }, { "query_id": "0f47b943-0225-4d71-af1d-01ee1cd0ab21", "query": "WANDB_WATCH all detailed visualization", "matching_document": "## Intro to Pytorch with W&B\n### Visualizing Results\nOne you\u2019ve trained your model you can visualize the predictions made by your model, its training and loss, gradients, best hyper-parameters and review associated code. \n\nTo view runs created by people in this public project: \n\n* Check out the .\n* Press 'option+space' to expand the runs table, comparing all the results from everyone who has tried this script.\n* Click on the name of a run to dive in deeper to that single run on its own run page. \n\n### Visualize Gradients \n\nClick through to a single run to see more details about that run. For example, on [this run page](https://web.archive.org/web/20210511070655/https://app.wandb.ai/wandb/pytorch-intro/runs/f8mmkoxt) you can see the gradients I logged when I ran this script. \n\n### Visualize Predictions \n\nYou can visualize predictions made at every step by clicking on the Media tab. Here we can see an example of true labels and predictions made by our model on the CIFAR dataset. \n\n### Review Code \n\n### Visualize Relationships", "matching_document_document_id": "d276638d-566c-4118-abd5-77523e6a0ebf", "query_weight": 0.8361290322580645 }, { "query_id": "1c16f7fc-654b-4b1e-a8a5-94350907822c", "query": "How to configure WANDB_WATCH='all' for detailed monitoring?", "matching_document": "## Hugging Face Transformers\n### Advanced features and FAQs\n#### Additional W&B settings\n| Environment Variable | Usage |\n| --- | --- |\n| `WANDB_PROJECT` | Give your project a name (`huggingface` by default) |\n| `WANDB_LOG_MODEL` | Log the model checkpoint as a W&B Artifact (`false` by default) * `false` (default): No model checkpointing\n* `checkpoint`: A checkpoint will be uploaded every args.save\\_steps (set in the Trainer's TrainingArguments).\n* `end`: The final model checkpoint will be uploaded at the end of training.\n|\n| `WANDB_WATCH` | Set whether you'd like to log your models gradients, parameters or neither* `false` (default): No gradient or parameter logging\n* `gradients`: Log histograms of the gradients\n* `all`: Log histograms of gradients and parameters\n|\n| `WANDB_DISABLED` | Set to `true` to disable logging entirely (`false` by default) |\n| `WANDB_SILENT` | Set to `true` to silence the output printed by wandb (`false` by default) | \n\n```\nWANDB\\_WATCH=all\nWANDB\\_SILENT=true\n\n``` \n\n```\n%env WANDB_WATCH=all\n%env WANDB_SILENT=true\n\n```", "matching_document_document_id": "a372dbd0-ce6a-43e8-80bb-7c5e6a154258", "query_weight": 0.8361290322580645 }, { "query_id": "61727d73-3ae3-4f75-a616-d740fd1bab05", "query": "What are the best practices for monitoring and improving LLM-powered apps using Weights & Biases?", "matching_document": "## Prompts for LLMs\n\nW&B Prompts is a suite of LLMOps tools built for the development of LLM-powered applications. Use W&B Prompts to visualize and inspect the execution flow of your LLMs, analyze the inputs and outputs of your LLMs, view the intermediate results and securely store and manage your prompts and LLM chain configurations. \n\n## Use Cases \n\nW&B Prompts provides several solutions for building and monitoring LLM-based apps. Software developers, prompt engineers, ML practitioners, data scientists, and other stakeholders working with LLMs need cutting-edge tools to: \n\n* Explore and debug LLM chains\u00a0and prompts with greater granularity.\n* Monitor and observe LLMs to better understand and evaluate performance, usage, and budgets. \n\n## Products \n\n### Traces \n\nW&B\u2019s LLM tool is called\u00a0*Traces*.\u00a0**Traces**\u00a0allow you to track and visualize the inputs and outputs, execution flow, model architecture, and any intermediate results of your LLM chains. \n\n### Weave \n\n### How it works \n\n## Integrations", "matching_document_document_id": "99fd9bc0-a4ff-4291-9e1c-c6aab2b9b31a", "query_weight": 1.207741935483871 }, { "query_id": "ed2fbd3c-bcd9-4ee9-a3e9-1be213daba76", "query": "Wandb Vega list plotting", "matching_document": "## Customization Steps: Edit the Vega Spec\n```\n{\n\"$schema\": \"https://vega.github.io/schema/vega-lite/v4.json\",\n\"description\": \"A simple histogram\",\n\"data\": {\n\"name\": \"wandb\"\n},\n\"selection\": {\n\"grid\": {\n\"type\": \"interval\", \"bind\": \"scales\"\n}\n},\n\"title\": \"${field:title}\",\n\"mark\": {\"type\": \"bar\", \"tooltip\": {\"content\": \"data\"}},\n\"encoding\": {\n\"x\": {\n\"bin\" : {\"binned\" : false, \"step\" : 0.025},\n\"type\": \"quantitative\",\n\"field\": \"${field:value}\"\n},\n\"y\": {\n\"aggregate\": \"count\",\n\"scale\" : {\"domain\" : [0, 50]},\n\"stack\": null\n},\n\"opacity\": {\"value\": 0.6},\n\"detail\": [{\"field\": \"name\"}, {\"field\": \"color\"}],\n\"color\": {\n\"type\": \"nominal\",\n\"field\": \"name\",\n\"scale\": {\"range\": {\"field\": \"color\"}}\n}\n}\n}\n``` \n\n* When you're happy with your changes, save the result so you can reuse it. I named my chart \"histogram_small_bins\" and made it publicly accessible so that everyone who reads this report can view charts of that type. I recommend this setting if you're sharing the report with anyone and want them to be able to see your charts.", "matching_document_document_id": "f9cfb724-5de3-4a36-8c9b-b8bb71e099dd", "query_weight": 0.8361290322580645 }, { "query_id": "d48bf033-0c92-4283-88a9-758d3903675c", "query": "What are the common mistakes when logging images with Wandb?", "matching_document": "\"\"\"Format images for logging to W&B.\n\n Arguments:\n data_or_path: (numpy array, string, io) Accepts numpy array of\n image data, or a PIL image. The class attempts to infer\n the data format and converts it.\n mode: (string) The PIL mode for an image. Most common are \"L\", \"RGB\",\n \"RGBA\". Full explanation at https://pillow.readthedocs.io/en/stable/handbook/concepts.html#modes\n caption: (string) Label for display of image.\n\n Note : When logging a `torch.Tensor` as a `wandb.Image`, images are normalized. If you do not want to normalize your images, please convert your tensors to a PIL Image.\n\n Examples:\n ### Create a wandb.Image from a numpy array\n \n ```python\n import numpy as np\n import wandb\n\n with wandb.init() as run:\n examples = []\n for i in range(3):\n pixels = np.random.randint(low=0, high=256, size=(100, 100, 3))\n image = wandb.Image(pixels, caption=f\"random field {i}\")\n examples.append(image)\n run.log({\"examples\": examples})\n ```", "matching_document_document_id": "d99652d5-58ec-4116-995f-69388c97277f", "query_weight": 0.9754838709677419 }, { "query_id": "decdf9de-b60a-4aba-b8bd-2fe9739f6f39", "query": "debugging WandbLogger errors", "matching_document": "--- \n\n## displayed\\_sidebar: default \n\n# PyTorch Lightning \n\n**Try in a Colab Notebook here \u2192** \n\nPyTorch Lightning provides a lightweight wrapper for organizing your PyTorch code and easily adding advanced features such as distributed training and 16-bit precision. W&B provides a lightweight wrapper for logging your ML experiments. But you don't need to combine the two yourself: Weights & Biases is incorporated directly into the PyTorch Lightning library via the **`WandbLogger`**. \n\n## \u26a1 Get going lightning-fast with just two lines. \n\n```\nfrom pytorch\\_lightning.loggers import WandbLogger\nfrom pytorch\\_lightning import Trainer\n\nwandb\\_logger = WandbLogger(log\\_model=\"all\")\ntrainer = Trainer(logger=wandb\\_logger)\n\n``` \n\n:::info\n**Using wandb.log():** Please note that the `WandbLogger` logs to W&B using the Trainer's `global_step`. If you are making additional calls to `wandb.log` directly in your code, **do not** use the `step` argument in `wandb.log()`. \n\n## Sign up and Log in to wandb", "matching_document_document_id": "adf86265-a5aa-4a7c-9d12-af46a7fcd872", "query_weight": 0.9754838709677419 }, { "query_id": "3b189f4e-c907-430c-acd6-95326d82ebb3", "query": "Logging lists in Wandb", "matching_document": "Logged values don't have to be scalars. Logging any wandb object is supported.\n For example `wandb.log({\"example\": wandb.Image(\"myimage.jpg\")})` will log an\n example image which will be displayed nicely in the W&B UI.\n See the [reference documentation](https://docs.wandb.com/ref/python/data-types)\n for all of the different supported types or check out our\n [guides to logging](https://docs.wandb.ai/guides/track/log) for examples,\n from 3D molecular structures and segmentation masks to PR curves and histograms.\n `wandb.Table`s can be used to logged structured data. See our\n [guide to logging tables](https://docs.wandb.ai/guides/data-vis/log-tables)\n for details.\n\n Logging nested metrics is encouraged and is supported in the W&B UI.\n If you log with a nested dictionary like `wandb.log({\"train\":\n {\"acc\": 0.9}, \"val\": {\"acc\": 0.8}})`, the metrics will be organized into\n `train` and `val` sections in the W&B UI.", "matching_document_document_id": "f26fd629-7649-4b5c-83de-04164100d66a", "query_weight": 0.9754838709677419 }, { "query_id": "86c9d925-6a5c-48be-8704-00882fbb0993", "query": "Troubleshooting artifact path issues in W&B", "matching_document": "## Storage\n\nArtifact Storage \n\nW&B stores artifact files in a private Google Cloud Storage bucket located in the United States by default. All files are encrypted at rest and in transit. \n\nFor sensitive files, we recommend you set up Private Hosting or use reference artifacts. \n\nDuring training, W&B locally saves logs, artifacts, and configuration files in the following local directories: \n\n| File | Default location | To change default location set: |\n| --- | --- | --- |\n| logs | `./wandb` | `dir` in `wandb.init` or set the `WANDB_DIR` environment variable |\n| artifacts | `~/.cache/wandb` | the `WANDB_CACHE_DIR` environment variable |\n| configs | `~/.config/wandb` | the `WANDB_CONFIG_DIR` environment variable | \n\n:::caution\nDepending on the machine on `wandb` is initialized on, these default folders may not be located in a writeable part of the file system. This might trigger an error.\n::: \n\n### Clean up local artifact cache \n\n### How much storage does each artifact version use?", "matching_document_document_id": "2ec0a152-2b53-4183-8398-b61730217aac", "query_weight": 1.3935483870967742 }, { "query_id": "0799b584-d776-4e7d-908f-0bb273c06dc7", "query": "How does W&B handle artifact versioning after deletion?", "matching_document": "## Delete artifacts\n:::note\nArtifacts that are scheduled for deletion with a TTL policy, deleted with the W&B SDK, or deleted with the W&B App UI are first soft-deleted. Artifacts that are soft deleted undergo garbage collection before they are hard-deleted.\n::: \n\n### Delete an artifact version \n\nTo delete an artifact version: \n\n1. Select the name of the artifact. This will expand the artifact view and list all the artifact versions associated with that artifact.\n2. From the list of artifacts, select the artifact version you want to delete.\n3. On the right hand side of the workspace, select the kebab dropdown.\n4. Choose Delete. \n\nAn artifact version can also be deleted programatically via the delete() method. See the examples below. \n\n### Delete multiple artifact versions with aliases \n\nThe following code example demonstrates how to delete artifacts that have aliases associated with them. Provide the entity, project name, and run ID that created the artifacts. \n\n### Delete multiple artifact versions with a specific alias", "matching_document_document_id": "f4d54671-aed9-4d97-96af-8780b8507acd", "query_weight": 1.3935483870967742 }, { "query_id": "fa478d80-1303-4df4-8979-4ca7864191dd", "query": "API guide for Weights & Biases sweeps", "matching_document": "## Sweep\n\nView source on GitHub \n\nA set of runs associated with a sweep. \n\n```\nSweep(\nclient, entity, project, sweep\\_id, attrs=None\n)\n\n``` \n\n#### Examples: \n\nInstantiate with: \n\n```\napi = wandb.Api()\nsweep = api.sweep(path/to/sweep)\n\n``` \n\n| Attributes | |\n| --- | --- |\n| `runs` | (`Runs`) list of runs |\n| `id` | (str) sweep id |\n| `project` | (str) name of project |\n| `config` | (str) dictionary of sweep configuration |\n| `state` | (str) the state of the sweep |\n| `expected_run_count` | (int) number of expected runs for the sweep | \n\n## Methods \n\n### `best_run` \n\nView source \n\n```\nbest\\_run(\norder=None\n)\n\n``` \n\nReturn the best run sorted by the metric defined in config or the order passed in. \n\n### `display` \n\nView source \n\n```\ndisplay(\nheight=420, hidden=(False)\n) -> bool\n\n``` \n\nDisplay this object in jupyter. \n\n### `get` \n\nView source \n\n```\n@classmethod\nget(\nclient, entity=None, project=None, sid=None, order=None, query=None, \\*\\*kwargs\n)\n\n``` \n\n### `load` \n\nView source", "matching_document_document_id": "22d1890a-6b8a-47e3-93e5-47ad68eb9c99", "query_weight": 0.5574193548387096 }, { "query_id": "03a9c687-8c5f-4e99-ae5e-dadbfd5317fe", "query": "Single filter usage in Weights & Biases for sweep-related runs", "matching_document": "def runs(\n self,\n path: Optional[str] = None,\n filters: Optional[Dict[str, Any]] = None,\n order: str = \"-created_at\",\n per_page: int = 50,\n include_sweeps: bool = True,\n ):\n \"\"\"Return a set of runs from a project that match the filters provided.\n\n You can filter by `config.*`, `summary_metrics.*`, `tags`, `state`, `entity`, `createdAt`, etc.\n\n Examples:\n Find runs in my_project where config.experiment_name has been set to \"foo\"\n ```\n api.runs(path=\"my_entity/my_project\", filters={\"config.experiment_name\": \"foo\"})\n ```\n\n Find runs in my_project where config.experiment_name has been set to \"foo\" or \"bar\"\n ```\n api.runs(\n path=\"my_entity/my_project\",\n filters={\"$or\": [{\"config.experiment_name\": \"foo\"}, {\"config.experiment_name\": \"bar\"}]}\n )\n ```", "matching_document_document_id": "2e086565-4fcc-49b7-b84a-7a900033da51", "query_weight": 0.5574193548387096 }, { "query_id": "2788519e-b080-47f2-bb42-5279953d3f54", "query": "Where can I find examples of sweep configurations for Weights & Biases?", "matching_document": "## Quickstart\n### Define the search space with a sweep configuration\nWithin a dictionary, specify what hyperparameters you want to sweep over and. For more information about configuration options, see Define sweep configuration. \n\nThe proceeding example demonstrates a sweep configuration that uses a random search (`'method':'random'`). The sweep will randomly select a random set of values listed in the configuration for the batch size, epoch, and the learning rate. \n\nThroughout the sweeps, W&B will maximize the metric specified in the metric key (`metric`). In the following example, W&B will maximize (`'goal':'maximize'`) the validation accuracy (`'val_acc'`). \n\n```\n# 2: Define the search space\nsweep\\_configuration = {\n'method': 'random',\n'metric': {'goal': 'minimize', 'name': 'score'},\n'parameters':\n{\n'x': {'max': 0.1, 'min': 0.01},\n'y': {'values': [1, 3, 7]},\n}\n}\n\n```", "matching_document_document_id": "7dea52bc-7acf-4a8b-9f5d-07d5b0e8e240", "query_weight": 0.5574193548387096 }, { "query_id": "eb4432b1-1265-47fa-a5e4-230fbcebe1fb", "query": "What methods are available in the Weights & Biases API for accessing sweep results?", "matching_document": "## Api\n### Methods\n#### `sweep`\n\nView source \n\n```\nsweep(\npath=\"\"\n)\n\n``` \n\nReturn a sweep by parsing path in the form `entity/project/sweep_id`. \n\n| Arguments | |\n| --- | --- |\n| `path` | (str, optional) path to sweep in the form entity/project/sweep\\_id. If `api.entity` is set, this can be in the form project/sweep\\_id and if `api.project` is set this can just be the sweep\\_id. | \n\n| Returns | |\n| --- | --- |\n| A `Sweep` object. | |", "matching_document_document_id": "f931f42c-eb28-43ef-99fe-7e40be3f5fa4", "query_weight": 0.5574193548387096 }, { "query_id": "9988b1e8-3955-4787-9621-6d226506718a", "query": "Optimizing hyperparameter tuning with parallel sweeps", "matching_document": "## Parallelize agents\n\nParallelize agents \n\nParallelize your W&B Sweep agents on a multi-core or multi-GPU machine. Before you get started, ensure you have initialized your W&B Sweep. For more information on how to initialize a W&B Sweep, see Initialize sweeps. \n\n### Parallelize on a multi-CPU machine \n\nDepending on your use case, explore the proceeding tabs to learn how to parallelize W&B Sweep agents using the CLI or within a Jupyter Notebook. \n\nUse the `wandb agent` command to parallelize your W&B Sweep agent across multiple CPUs with the terminal. Provide the sweep ID that was returned when you initialized the sweep. \n\n1. Open more than one terminal window on your local machine.\n2. Copy and paste the code snippet below and replace `sweep_id` with your sweep ID: \n\n```\nwandb agent sweep_id\n\n``` \n\n### Parallelize on a multi-GPU machine \n\nFollow the procedure outlined to parallelize your W&B Sweep agent across multiple GPUs with a terminal using CUDA Toolkit:", "matching_document_document_id": "88000644-cc6f-409b-92f0-2697b33b05b5", "query_weight": 0.8825806451612903 }, { "query_id": "195dd425-1d7e-4f4c-99d4-b93ebb0816f5", "query": "pruning hyperparameter combinations", "matching_document": "## Sweeps on Launch\n### Create a custom sweep scheduler\n```\n# optuna\\_config\\_basic.yaml\ndescription: A basic Optuna scheduler\njob: wandb/sweep-jobs/job-fashion-MNIST-train:latest\nrun\\_cap: 5\nmetric:\nname: epoch/val\\_loss\ngoal: minimize\n\nscheduler:\njob: wandb/sweep-jobs/job-optuna-sweep-scheduler:latest\nresource: local-container # required for scheduler jobs sourced from images\nnum\\_workers: 2\n\n# optuna specific settings\nsettings:\npruner:\ntype: PercentilePruner\nargs:\npercentile: 25.0 # kill 75% of runs\nn\\_warmup\\_steps: 10 # pruning disabled for first x steps\n\nparameters:\nlearning\\_rate:\nmin: 0.0001\nmax: 0.1\n``` \n\n3. Lastly, launch the sweep to an active queue with the launch-sweep command: \n\n```bash\nwandb launch-sweep -q -p -e \n``` \n\nFor the exact implementation of the Optuna sweep scheduler job, see wandb/launch-jobs. For more examples of what is possible with the Optuna scheduler, check out wandb/examples. \n\n\n", "matching_document_document_id": "dbd99861-821a-4f36-8add-fc595a861549", "query_weight": 0.8825806451612903 }, { "query_id": "6efa2b6b-fd34-4c40-a105-14253418ad1a", "query": "Managing GPU memory usage in W&B sweeps", "matching_document": "## Monitor & Improve GPU Usage for Model Training\n#### 1. Measure your GPU usage consistently over your entire training runs\nYou can\u2019t improve GPU usage without measuring it. It\u2019s not hard to take a snapshot of your usage with useful tools like nvidia-smi, but a simple way to find issues is to track usage over time. Anyone can turn on system monitoring in the background, which will track GPU, CPU, memory usage etc over time by adding two lines to their code: \n\n```\nimport wandb\nwandb.init()\n``` \n\nThe wandb.init() function will create a lightweight child process that will collect system metrics and send them to a wandb server where you can look at them and compare across runs with graphs like these: \n\nThe danger of taking a single measurement is that GPU usage can change over time. This is a common pattern we see where our user Boris is training an RNN; mid-training, his usage plummets from 80 percent to around 25 percent.", "matching_document_document_id": "f57cb60e-1cde-4855-a5e2-1688ac4755e5", "query_weight": 0.8825806451612903 }, { "query_id": "54ebe14b-5c16-4feb-8d1c-20f60be6da4c", "query": "Managing GPU memory in hyperparameter optimization with W&B", "matching_document": "## System Metrics \u2013 CPU and GPU Usage\n\nVisualize live metrics like GPU utilization to identify training bottlenecks and avoid wasting expensive resources with automatically generated system metrics. \n\nKeep GPU costs down, and increase iteration speeds by efficiently allocating resources in distributed training pipelines. \n\nwandb is framework agnostic, to start capturing your system metrics, simply add the following lines to your training script: \n\n```\nimport wandb\nwandb.init(project=\"my-project\")\n\n# model training code here\n\n``` \n\nInitializing wandb automatically logs system metrics every 2 seconds, averaged over a 30 second period. The metrics include: \n\n* CPU Utilization\n* System Memory Utilization\n* Disk I/O Utilization\n* Network traffic (bytes sent and received)\n* GPU Utilization\n* GPU Temperature\n* GPU Time Spent Accessing Memory (as a percentage of the sample time)\n* GPU Memory Allocated", "matching_document_document_id": "189783f6-14a3-4d18-8c33-a868dd2cf8a8", "query_weight": 0.8825806451612903 }, { "query_id": "9af3101b-b151-4d2d-a8f1-0eb2db728520", "query": "wandb.init error", "matching_document": "## init\n#### Add metadata about the run to the config\n\nPass a dictionary-style object as the `config` keyword argument to add\nmetadata, like hyperparameters, to your run. \n\n```\nimport wandb\n\nconfig = {\"lr\": 3e-4, \"batch\\_size\": 32}\nconfig.update({\"architecture\": \"resnet\", \"depth\": 34})\nwandb.init(config=config)\n\n``` \n\n| Raises | |\n| --- | --- |\n| `Error` | if some unknown or internal error happened during the run initialization. |\n| `AuthenticationError` | if the user failed to provide valid credentials. |\n| `CommError` | if there was a problem communicating with the WandB server. |\n| `UsageError` | if the user provided invalid arguments. |\n| `KeyboardInterrupt` | if user interrupts the run. | \n\n| Returns | |\n| --- | --- |\n| A `Run` object. | |", "matching_document_document_id": "1974561f-5b98-4306-91a8-59f948714a10", "query_weight": 1.1148387096774193 }, { "query_id": "807a7c8f-7354-4455-996c-e3581f9a262c", "query": "What does Response 409 mean in W&B?", "matching_document": "@app.route(\"/graphql\", methods=[\"POST\"])\n @snoop.relay\n def graphql():\n # TODO: in tests wandb-username is set to the test name, lets scope ctx to it\n ctx = get_ctx()\n base_url = request.url_root.rstrip(\"/\")\n test_name = request.headers.get(\"X-WANDB-USERNAME\")\n if test_name:\n app.logger.info(\"Test request from: %s\", test_name)\n app.logger.info(\"graphql post\")\n\n if \"fail_graphql_times\" in ctx:\n if ctx[\"fail_graphql_count\"] < ctx[\"fail_graphql_times\"]:\n ctx[\"fail_graphql_count\"] += 1\n return json.dumps({\"errors\": [\"Server down\"]}), 500\n if \"rate_limited_times\" in ctx:\n if ctx[\"rate_limited_count\"] < ctx[\"rate_limited_times\"]:\n ctx[\"rate_limited_count\"] += 1\n return json.dumps({\"error\": \"rate limit exceeded\"}), 429\n if ctx[\"graphql_conflict\"]:\n return json.dumps({\"error\": \"resource already exists\"}), 409", "matching_document_document_id": "3e24346e-8605-49d1-87b4-6d41666ac0eb", "query_weight": 1.3470967741935482 }, { "query_id": "439fb925-a97d-45c5-ad1b-fa0194b3dac3", "query": "W&B external data integration", "matching_document": "## Better Models Faster with Weights & Biases\n### 4. Resource efficient model training\n\nGPU resources are expensive and we're trying to win our Kaggle competitions by being as resource efficient (read: cheap) as possible. W&B shows you how many resources you're using with this nifty little System Metrics tab. \n\nThis is useful because you can see which model experiments were the most resource intensive, and limit the types of models and hyperparameter values you try. For instance you might realize that doubling the batch size quadruples the GPU usage, which is no bueno. \n\nBy keeping track of system metrics, we can be time and resource efficient! And save the planet in a small way.", "matching_document_document_id": "673c9e4a-bcc2-4806-a789-dbf646d96273", "query_weight": 1.3470967741935482 }, { "query_id": "55294472-dc37-4497-b469-55d67802e467", "query": "Setup guide for wandb distributed training", "matching_document": "## Experiments FAQ\n#### How can I use wandb with multiprocessing, e.g. distributed training?\n\nIf your training program uses multiple processes you will need to structure your program to avoid making wandb method calls from processes where you did not run `wandb.init()`.\\\n\\\nThere are several approaches to managing multiprocess training: \n\n1. Call `wandb.init` in all your processes, using the group keyword argument to define a shared group. Each process will have its own wandb run and the UI will group the training processes together.\n2. Call `wandb.init` from just one process and pass data to be logged over multiprocessing queues. \n\n:::info\nCheck out the Distributed Training Guide for more detail on these two approaches, including code examples with Torch DDP.\n:::", "matching_document_document_id": "cc461aea-36f0-4a9b-be45-e7110b8cfd88", "query_weight": 0.7432258064516128 }, { "query_id": "1e958f82-d95b-4f61-8dce-964f02ac2a55", "query": "Best practices for using wandb in multi-GPU setups", "matching_document": "## Monitor & Improve GPU Usage for Model Training\n#### 1. Measure your GPU usage consistently over your entire training runs\nYou can see his complete set of stats and training log at [https://app.wandb.ai/borisd13/char-RNN/runs/cw9gnx9z/system](https://app.wandb.ai/borisd13/char-RNN/runs/cw9gnx9z/system)[](https://web.archive.org/web/20210406085446/https://app.wandb.ai/borisd13/char-RNN/runs/cw9gnx9z/system) \n\nA related case we commonly see with multiple GPUs is that mid-training, some of the GPUs stop handling any load. In this example both GPUs started off doing computations, but a few minutes in, all the load is sent to a single GPU. This could be intentional but this is often the sign of a hard to catch bug in the code. \n\nAnother common issue we see is that there are long periods of not using the GPUs - often corresponding with a testing or validation phases in training or bottlenecked on some data preprocessing. Here is a typical graph, training on 8 GPUs where all of them turn off and wait for some time at a regular interval.", "matching_document_document_id": "ef9bef39-695c-45e1-8e43-8a7681cd49eb", "query_weight": 0.7432258064516128 }, { "query_id": "b20da28d-64ca-4de9-993d-91a8f64f3bc1", "query": "Troubleshooting wandb distributed training logging", "matching_document": "In distributed training, you can either create a single run in the rank 0 process\n and then log information only from that process, or you can create a run in each process,\n logging from each separately, and group the results together with the `group` argument\n to `wandb.init`. For more details on distributed training with W&B, check out\n [our guide](https://docs.wandb.ai/guides/track/log/distributed-training).\n\n Currently, there is a parallel `Run` object in the `wandb.Api`. Eventually these\n two objects will be merged.\n\n Attributes:\n summary: (Summary) Single values set for each `wandb.log()` key. By\n default, summary is set to the last value logged. You can manually\n set summary to the best value, like max accuracy, instead of the\n final value.\n \"\"\"", "matching_document_document_id": "6fcdf0c3-8da7-4cbb-9a9b-3e958859f1c5", "query_weight": 0.7432258064516128 }, { "query_id": "a3bd22aa-e154-400d-b2da-81397aede6ae", "query": "Steps to change the owner of a project to a team in Weights & Biases.", "matching_document": "## Manage users\n### Manage your organization\n#### Change a user's organization-level role\n1. Navigate to the W&B Organization dashboard.\n2. Search for the user you want to modify in the search bar.\n3. Hover your mouse to the **Role** column. Click on the pencil icon that appears.\n4. From the dropdown, select a different role you want to assign.", "matching_document_document_id": "a1d6d930-6788-4fdf-93e9-e0137460c439", "query_weight": 2.136774193548387 }, { "query_id": "b6bfebc5-c25e-437e-ae22-5350a789ee8c", "query": "access runs without GUI", "matching_document": "## Api\n### Methods\n#### `run`\n\nView source \n\n```\nrun(\npath=\"\"\n)\n\n``` \n\nReturn a single run by parsing path in the form entity/project/run\\_id. \n\n| Arguments | |\n| --- | --- |\n| `path` | (str) path to run in the form `entity/project/run_id`. If `api.entity` is set, this can be in the form `project/run_id` and if `api.project` is set this can just be the run\\_id. | \n\n| Returns | |\n| --- | --- |\n| A `Run` object. | |", "matching_document_document_id": "0b765ba4-fcba-45b7-ab5a-5088d345c06a", "query_weight": 2.136774193548387 }, { "query_id": "bcc8403e-5aa9-4c89-b4b4-b53f6140e763", "query": "sharing permissions", "matching_document": "## Collaborate on reports\n\nCollaborate and Share W&B Reports \n\nOnce you have saved a report, you can select the **Share** button to collaborate. A draft copy of the report is created when you select the **Edit** button. Draft reports auto-save. Select **Save to report** to publish your changes to the shared report. \n\nA warning notification will appear if an edit conflict occurs. This can occur if you and another collaborator edit the same report at the same time. The warning notification will guide you to resolve potential edit conflicts. \n\n### Comment on reports \n\nClick the comment button on a panel in a report to add a comment directly to that panel. \n\n### Who can edit and share reports? \n\nReports that are created within an individual's private project is only visible to that user. The user can share their project to a team or to the public.", "matching_document_document_id": "5bab537b-05eb-4ac3-9db7-27a98adcbed0", "query_weight": 2.136774193548387 }, { "query_id": "68e39df9-4d58-47b5-bd74-08a0110e1c16", "query": "share personal project", "matching_document": "--- \n\n## description: Collaborate and share W&B Reports with peers, co-workers, and your team. \n\n# Collaborate on reports \n\nCollaborate and Share W&B Reports \n\nOnce you have saved a report, you can select the **Share** button to collaborate. A draft copy of the report is created when you select the **Edit** button. Draft reports auto-save. Select **Save to report** to publish your changes to the shared report. \n\nA warning notification will appear if an edit conflict occurs. This can occur if you and another collaborator edit the same report at the same time. The warning notification will guide you to resolve potential edit conflicts. \n\n### Comment on reports \n\nClick the comment button on a panel in a report to add a comment directly to that panel. \n\n### Who can edit and share reports? \n\nReports that are created within an individual's private project is only visible to that user. The user can share their project to a team or to the public.", "matching_document_document_id": "7fe941de-f8fe-4acc-b460-004c8b6a298e", "query_weight": 2.136774193548387 }, { "query_id": "9ccc6dff-fda9-4a7f-b02f-686cc199e120", "query": "How can a user gain admin rights in their own academic team on Weights & Biases?", "matching_document": "## Teams\n### Team Roles and Permissions\n* **Admin**: Team admins can add and remove other admins or team members. They have permissions to modify all projects and full deletion permissions. This includes, but is not limited to, deleting runs, projects, artifacts, and sweeps.\n* **Member**: A regular member of the team. A team member is invited by email by the team admin. A team member cannot invite other members. Team members can only delete runs and sweep runs created by that member. Suppose you have two members A and B. Member B moves a Run from team B's project to a different project owned by Member A. Member A can not delete the Run Member B moved to Member A's project. Only the member that creates the Run, or the team admin, can delete the run.\n* **Service**: A service worker, an API key useful for using W&B with your run automation tools. If you use the API key from a service account for your team, make sure to set the environment variable **WANDB\\_USERNAME** to attribute runs to the correct user.\n* **View-Only (Enterprise-only feature)**: View-Only members can view assets within the team such as runs, reports, and workspaces. They can follow and comment on reports, but they can not create, edit, or delete project overview, reports, or runs. View-Only members do not have an API key.", "matching_document_document_id": "a7c2233e-d024-4e8b-98ef-bc9af7bc6934", "query_weight": 2.136774193548387 }, { "query_id": "c6e49fe5-2739-4bb8-97ed-37b2d49f7831", "query": "Weights & Biases artifacts large datasets management", "matching_document": "## Storage\n\nIf you are approaching or exceeding your storage limit, there are multiple paths forward to manage your data. The path that's best for you will depend on your account type and your current project setup. \n\n## Manage storage consumption \n\nW&B offers different methods of optimizing your storage consumption: \n\n* Use\u00a0reference artifacts\u00a0to track files saved outside the W&B system, instead of uploading them to W&B storage.\n* Use an external cloud storage bucket for storage. *(Enterprise only)* \n\n## Delete data \n\nYou can also choose to delete data to remain under your storage limit. There are several ways to do this: \n\n* Delete data interactively with the app UI.\n* Set a TTL policy on Artifacts so they are automatically deleted.", "matching_document_document_id": "6dea2d7d-58f2-4830-8dc1-27b3bf719859", "query_weight": 0.5109677419354839 }, { "query_id": "23d2cf7c-b1fe-434d-9172-6d0e9cab67cc", "query": "What are common reasons for wandb.log to display a table instead of the expected chart visualization?", "matching_document": "## Log Media and Objects in Experiments\n### What data is logged with specific W&B API calls?\n* **Datasets**: You have to specifically log images or other dataset samples for them to stream to W&B.\n* **Plots**: Use `wandb.plot` with `wandb.log` to track charts. See Log Plots for more information.\n* **Tables**: Use `wandb.Table` to log data to visualize and query with W&B. See Log Tables for more information.\n* **PyTorch gradients**: Add `wandb.watch(model)` to see gradients of the weights as histograms in the UI.\n* **Configuration information**: Log hyperparameters, a link to your dataset, or the name of the architecture you're using as config parameters, passed in like this: `wandb.init(config=your_config_dictionary)`. See the PyTorch Integrations page for more information.\n* **Metrics**: Use `wandb.log` to see metrics from your model. If you log metrics like accuracy and loss from inside your training loop, you'll get live updating graphs in the UI.", "matching_document_document_id": "f18c8e83-0e30-4a22-89a3-d60f135b6a60", "query_weight": 0.8361290322580645 }, { "query_id": "beb0a04f-731f-4c45-b05b-7866391a8a0b", "query": "Examples of using artifact.add_file and wandb.save in machine learning projects", "matching_document": "## `.add`, `.add_file`, `.add_dir`, and `.add_reference`\n### Create a new artifact version from an existing version\n```\ndraft\\_artifact.save()\n\n``` \n\nPutting it all together, the code examples above look like: \n\n```\nwith wandb.init(job\\_type=\"modify dataset\") as run:\nsaved\\_artifact = run.use\\_artifact(\n\"my\\_artifact:latest\"\n) # fetch artifact and input it into your run\ndraft\\_artifact = saved\\_artifact.new\\_draft() # create a draft version\n\n# modify a subset of files in the draft version\ndraft\\_artifact.add\\_file(\"file\\_to\\_add.txt\")\ndraft\\_artifact.remove(\"dir\\_to\\_remove/\")\nrun.log\\_artifact(\nartifact\n) # log your changes to create a new version and mark it as output to your run\n\n``` \n\n```\nclient = wandb.Api()\nsaved\\_artifact = client.artifact(\"my\\_artifact:latest\") # load your artifact\ndraft\\_artifact = saved\\_artifact.new\\_draft() # create a draft version\n\n# modify a subset of files in the draft version\ndraft\\_artifact.remove(\"deleted\\_file.txt\")\ndraft\\_artifact.add\\_file(\"modified\\_file.txt\")\ndraft\\_artifact.save() # commit changes to the draft\n\n```", "matching_document_document_id": "59663044-361c-473c-9875-595988b69e24", "query_weight": 0.7432258064516128 }, { "query_id": "56a5d203-a519-457b-8ba1-bb1bb33dff77", "query": "Difference between artifact.add_file and wandb.save in Weights & Biases documentation", "matching_document": "## `.add`, `.add_file`, `.add_dir`, and `.add_reference`\nartifact.add_file(\"image1.png\")\nrun.log_artifact(artifact) \n\n```\n\n`\n\n\nCreate an artifact version outside of a W&B run:\n\n1. Create a new artifact or retrieve an existing one with `wanb.Artifact`. (Line 1)\n2. Add files to the artifact with `.add_file`. (Line 4)\n3. Save the artifact with `.save`. (Line 5)`python showLineNumbers\nartifact = wandb.Artifact(\"artifact\\_name\", \"artifact\\_type\")\n\n# Add Files and Assets to the artifact using\n\n# `.add`, `.add_file`, `.add_dir`, and `.add_reference`\n\nartifact.add\\_file(\"image1.png\")\nartifact.save()\n``` \n\n### Distributed runs \n\nAllow a collection of runs to collaborate on a version before committing it. This is in contrast to single run mode described above where one run provides all the data for a new version. \n\n## Create a new artifact version from an existing version", "matching_document_document_id": "eadf7453-ea3b-4447-9f02-40afdc16faa8", "query_weight": 0.7432258064516128 }, { "query_id": "5e17179a-aa1f-4956-a72e-cea7ecf92ab7", "query": "how can I save codes into artifacts? I have tried the methods offered here:\nhttps://docs.wandb.ai/guides/app/features/panels/code\nhttps://docs.wandb.ai/ref/python/run#log_code\nBut I can only save codes from the file calling `wandb.init()`, I still cannot save other files. Is there a case to show how to really do this?", "matching_document": "## Run\n### Methods\n#### `log_code`\n| Arguments | |\n| --- | --- |\n| `root` | The relative (to `os.getcwd()`) or absolute path to recursively find code from. |\n| `name` | (str, optional) The name of our code artifact. By default, we'll name the artifact `source-$PROJECT_ID-$ENTRYPOINT_RELPATH`. There may be scenarios where you want many runs to share the same artifact. Specifying name allows you to achieve that. |\n| `include_fn` | A callable that accepts a file path and (optionally) root path and returns True when it should be included and False otherwise. This defaults to: `lambda path, root: path.endswith(\".py\")` |\n| `exclude_fn` | A callable that accepts a file path and (optionally) root path and returns `True` when it should be excluded and `False` otherwise. This defaults to a function that excludes all files within `<root>/.wandb/` and `<root>/wandb/` directories. | \n\n#### Examples: \n\nBasic usage \n\n```\nrun.log\\_code()\n\n``` \n\nAdvanced usage", "matching_document_document_id": "1c1ab624-25b4-465c-ac9d-75d589ae2177", "query_weight": 0.7432258064516128 }, { "query_id": "19d0f554-0d4a-468b-ae09-eb92405aba4b", "query": "Implementing login mechanisms for SageMaker estimator", "matching_document": "def sagemaker_auth(overrides=None, path=\".\", api_key=None):\n \"\"\"Write a secrets.env file with the W&B ApiKey and any additional secrets passed.\n\n Arguments:\n overrides (dict, optional): Additional environment variables to write\n to secrets.env\n path (str, optional): The path to write the secrets file.\n \"\"\"\n settings = wandb.setup().settings\n current_api_key = wandb.wandb_lib.apikey.api_key(settings=settings)\n\n overrides = overrides or dict()\n api_key = overrides.get(env.API_KEY, api_key or current_api_key)\n if api_key is None:\n raise ValueError(\n \"Can't find W&B ApiKey, set the WANDB_API_KEY env variable \"\n \"or run `wandb login`\"\n )\n overrides[env.API_KEY] = api_key\n with open(os.path.join(path, \"secrets.env\"), \"w\") as file:\n for k, v in overrides.items():\n file.write(f\"{k}={v}\\n\")", "matching_document_document_id": "f7eeca6b-023b-4cae-a525-33202a2e8237", "query_weight": 1.0683870967741935 }, { "query_id": "9a76ce7e-0616-4ec5-8d28-74c7db47e901", "query": "secure SageMaker estimator training", "matching_document": "## Using AWS Sagemaker and Weights & Biases Together on Digit Recognition with MNIST\n### Integrating Weights & Biases to AWS SageMaker\n2) Pass Weights & Biases API Key to the Estimator \n\nGo to [https://wandb.ai](https://wandb.ai/) to signup for a free account, after which you could go to [https://wandb.ai/authorize](https://wandb.ai/authorize) to get your API key! \n\nYou should see something like the screenshot above. Simply copy the API key and paste it in the following code block. \n\n```\nestimator = PyTorch(entry_point='mnist.py',\nsource_dir=\"src\",\nrole=role,\npy_version='py3',\nframework_version='1.8.0',\ninstance_count=1,\ninstance_type='ml.c5.2xlarge',\nhyperparameters={\n'epochs': 1,\n'backend': 'gloo'\n},\n# Pass Weights & Biases API Key as an environment variable\n# You can find the key by going to https://wandb.ai/authorize\nenvironment={\"WANDB_API_KEY\": current_api_key})\n``` \n\nFirst, run the following to train the estimator: \n\n```\nestimator.fit({'training': inputs})\n``` \n\nThen, you'll see your W&B run in your dashboard. From here, you can check out a variety of metrics such as losses, system metrics, and gradient histograms:", "matching_document_document_id": "1ce62199-dbfc-4dc2-98f0-32ac40659f17", "query_weight": 1.0683870967741935 }, { "query_id": "4c158272-668f-4708-89c6-23a53ca066b0", "query": "What are the permissions required for sharing W&B reports publicly?", "matching_document": "--- \n\n## description: Collaborate and share W&B Reports with peers, co-workers, and your team. \n\n# Collaborate on reports \n\nCollaborate and Share W&B Reports \n\nOnce you have saved a report, you can select the **Share** button to collaborate. A draft copy of the report is created when you select the **Edit** button. Draft reports auto-save. Select **Save to report** to publish your changes to the shared report. \n\nA warning notification will appear if an edit conflict occurs. This can occur if you and another collaborator edit the same report at the same time. The warning notification will guide you to resolve potential edit conflicts. \n\n### Comment on reports \n\nClick the comment button on a panel in a report to add a comment directly to that panel. \n\n### Who can edit and share reports? \n\nReports that are created within an individual's private project is only visible to that user. The user can share their project to a team or to the public.", "matching_document_document_id": "7fe941de-f8fe-4acc-b460-004c8b6a298e", "query_weight": 0.6038709677419355 }, { "query_id": "7c05beed-3754-469b-a3be-cebb25892af8", "query": "Can public cloud customers make W&B reports accessible to their colleagues without W&B access?", "matching_document": "## Anonymous Mode\n\nAre you publishing code that you want anyone to be able to run easily? Use Anonymous Mode to let someone run your code, see a W&B dashboard, and visualize results without needing to create a W&B account first. \n\nAllow results to be logged in Anonymous Mode with `wandb.init(`**`anonymous=\"allow\"`**`)` \n\n:::info\n**Publishing a paper?** Please cite W&B, and if you have questions about how to make your code accessible while using W&B, reach out to us at support@wandb.com.\n::: \n\n### How does someone without an account see results? \n\nIf someone runs your script and you have to set `anonymous=\"allow\"`: \n\n### What happens to users with existing accounts? \n\nIf you set `anonymous=\"allow\"` in your script, we will check to make sure there's not an existing account first, before creating an anonymous account. This means that if a W&B user finds your script and runs it, their results will be logged correctly to their account, just like a normal run. \n\n## Example usage", "matching_document_document_id": "b16c8971-a81d-4473-a2b7-81b03075d7c4", "query_weight": 0.6038709677419355 }, { "query_id": "b4837ef2-c8df-4790-a914-8d94e38d2dbc", "query": "optimizing sweep execution", "matching_document": "## Parallelize agents\n\nParallelize agents \n\nParallelize your W&B Sweep agents on a multi-core or multi-GPU machine. Before you get started, ensure you have initialized your W&B Sweep. For more information on how to initialize a W&B Sweep, see Initialize sweeps. \n\n### Parallelize on a multi-CPU machine \n\nDepending on your use case, explore the proceeding tabs to learn how to parallelize W&B Sweep agents using the CLI or within a Jupyter Notebook. \n\nUse the `wandb agent` command to parallelize your W&B Sweep agent across multiple CPUs with the terminal. Provide the sweep ID that was returned when you initialized the sweep. \n\n1. Open more than one terminal window on your local machine.\n2. Copy and paste the code snippet below and replace `sweep_id` with your sweep ID: \n\n```\nwandb agent sweep_id\n\n``` \n\n### Parallelize on a multi-GPU machine \n\nFollow the procedure outlined to parallelize your W&B Sweep agent across multiple GPUs with a terminal using CUDA Toolkit:", "matching_document_document_id": "88000644-cc6f-409b-92f0-2697b33b05b5", "query_weight": 1.207741935483871 }, { "query_id": "ecdb64b4-3560-416b-875e-596c6a1d30f3", "query": "GPU utilization in sweeps", "matching_document": "## Parallelize agents\n#### Parallelize on a multi-GPU machine\n1. Open more than one terminal window on your local machine.\n2. Specify the GPU instance to use with `CUDA_VISIBLE_DEVICES` when you start a W&B Sweep job (`wandb agent`). Assign `CUDA_VISIBLE_DEVICES` an integer value corresponding to the GPU instance to use. \n\nFor example, suppose you have two NVIDIA GPUs on your local machine. Open a terminal window and set `CUDA_VISIBLE_DEVICES` to `0` (`CUDA_VISIBLE_DEVICES=0`). Replace `sweep_ID` in the proceeding example with the W&B Sweep ID that is returned when you initialized a W&B Sweep: \n\nTerminal 1 \n\n```\nCUDA\\_VISIBLE\\_DEVICES=0 wandb agent sweep_ID\n\n``` \n\nOpen a second terminal window. Set `CUDA_VISIBLE_DEVICES` to `1` (`CUDA_VISIBLE_DEVICES=1`). Paste the same W&B Sweep ID for the `sweep_ID` mentioned in the proceeding code snippet: \n\nTerminal 2 \n\n```\nCUDA\\_VISIBLE\\_DEVICES=1 wandb agent sweep_ID\n\n```", "matching_document_document_id": "03228a82-a0d9-405b-ba85-e645187827e1", "query_weight": 1.207741935483871 }, { "query_id": "a1fa708f-b186-481c-9098-338c43082926", "query": "What settings are required to run multiple agents in a sweep simultaneously?", "matching_document": "## Parallelize agents\n#### Parallelize on a multi-GPU machine\n1. Open more than one terminal window on your local machine.\n2. Specify the GPU instance to use with `CUDA_VISIBLE_DEVICES` when you start a W&B Sweep job (`wandb agent`). Assign `CUDA_VISIBLE_DEVICES` an integer value corresponding to the GPU instance to use. \n\nFor example, suppose you have two NVIDIA GPUs on your local machine. Open a terminal window and set `CUDA_VISIBLE_DEVICES` to `0` (`CUDA_VISIBLE_DEVICES=0`). Replace `sweep_ID` in the proceeding example with the W&B Sweep ID that is returned when you initialized a W&B Sweep: \n\nTerminal 1 \n\n```\nCUDA\\_VISIBLE\\_DEVICES=0 wandb agent sweep_ID\n\n``` \n\nOpen a second terminal window. Set `CUDA_VISIBLE_DEVICES` to `1` (`CUDA_VISIBLE_DEVICES=1`). Paste the same W&B Sweep ID for the `sweep_ID` mentioned in the proceeding code snippet: \n\nTerminal 2 \n\n```\nCUDA\\_VISIBLE\\_DEVICES=1 wandb agent sweep_ID\n\n```", "matching_document_document_id": "03228a82-a0d9-405b-ba85-e645187827e1", "query_weight": 1.207741935483871 }, { "query_id": "b75236ec-7158-433c-b848-d23fb4eded7f", "query": "How do I run multiple instances of a sweep across multiple GPUs, note that I didn't configure the sweep through a yaml file.", "matching_document": "## Multi-GPU Hyperparameter Sweeps in Three Simple Steps\nSetting up the infrastructure for these sweeps can be tedious. So we've built W&B sweeps to be simple to set up and flexible to deploy. Inspired by [Google's Vizier](https://web.archive.org/web/20210501083721/https://ai.google/research/pubs/pub46180), we've implemented a wide range of features, including bayesian optimization and [hyperband](https://web.archive.org/web/20210501083721/https://arxiv.org/abs/1603.06560) early stopping. Integration is simple: if you have a machine learning script running on the command line, you\u2019re ready to go. \n\n### Step 1: Select Hyperparameters \n\nFirst, you\u2019ll want to select the hyperparameters you\u2019re sweeping over. Set this up in a YAML file, as detailed further in the sweep docs. \n\n```\nwandb init # Initialize your project repo\nwandb sweep sweep.yaml # returns your SWEEP_ID\n``` \n\n### Step 2: Launch Agents \n\nGrab your sweep ID from the output of the command above and launch some agents to begin running your sweep. \n\n```\nwandb agent mcg70107\n```", "matching_document_document_id": "32fdc30f-fe0c-4b05-8419-67aa9837edb9", "query_weight": 1.207741935483871 }, { "query_id": "331f70d4-5a15-4c85-93f8-56b6c9139e4a", "query": "How can I log molecules to W&B? Is there any limitation of logging molecules to W&B?", "matching_document": "# Log points and boxes in W&B\npoint\\_scene = wandb.Object3D(\n{\n\"type\": \"lidar/beta\",\n\"points\": np.array( # add points, as in a point cloud\n[[0.4, 1, 1.3], [1, 1, 1], [1.2, 1, 1.2]]\n),\n\"boxes\": np.array( # draw 3d boxes\n[\n{\n\"corners\": [\n[0, 0, 0],\n[0, 1, 0],\n[0, 0, 1],\n[1, 0, 0],\n[1, 1, 0],\n[0, 1, 1],\n[1, 0, 1],\n[1, 1, 1],\n],\n\"label\": \"Box\",\n\"color\": [123, 321, 111],\n},\n{\n\"corners\": [\n[0, 0, 0],\n[0, 2, 0],\n[0, 0, 2],\n[2, 0, 0],\n[2, 2, 0],\n[0, 2, 2],\n[2, 0, 2],\n[2, 2, 2],\n],\n\"label\": \"Box-2\",\n\"color\": [111, 321, 0],\n},\n]\n),\n\"vectors\": np.array( # add 3d vectors\n[{\"start\": [0, 0, 0], \"end\": [0.1, 0.2, 0.5]}]\n),\n}\n)\nwandb.log({\"point\\_scene\": point\\_scene})\n\n``` \n\n```\nwandb.log({\"protein\": wandb.Molecule(\"6lu7.pdb\")})\n\n``` \n\nLog molecular data in any of 10 file types:`pdb`, `pqr`, `mmcif`, `mcif`, `cif`, `sdf`, `sd`, `gro`, `mol2`, or `mmtf.` \n\nW&B also supports logging molecular data from SMILES strings, `rdkit` `mol` files, and `rdkit.Chem.rdchem.Mol` objects.", "matching_document_document_id": "b701ff5d-ee83-4f78-94ff-bab9439bdf4b", "query_weight": 1.2541935483870967 }, { "query_id": "11a677ac-bee7-4352-ac94-3b029a7e6925", "query": "logging molecules W&B", "matching_document": "# Log points and boxes in W&B\npoint\\_scene = wandb.Object3D(\n{\n\"type\": \"lidar/beta\",\n\"points\": np.array( # add points, as in a point cloud\n[[0.4, 1, 1.3], [1, 1, 1], [1.2, 1, 1.2]]\n),\n\"boxes\": np.array( # draw 3d boxes\n[\n{\n\"corners\": [\n[0, 0, 0],\n[0, 1, 0],\n[0, 0, 1],\n[1, 0, 0],\n[1, 1, 0],\n[0, 1, 1],\n[1, 0, 1],\n[1, 1, 1],\n],\n\"label\": \"Box\",\n\"color\": [123, 321, 111],\n},\n{\n\"corners\": [\n[0, 0, 0],\n[0, 2, 0],\n[0, 0, 2],\n[2, 0, 0],\n[2, 2, 0],\n[0, 2, 2],\n[2, 0, 2],\n[2, 2, 2],\n],\n\"label\": \"Box-2\",\n\"color\": [111, 321, 0],\n},\n]\n),\n\"vectors\": np.array( # add 3d vectors\n[{\"start\": [0, 0, 0], \"end\": [0.1, 0.2, 0.5]}]\n),\n}\n)\nwandb.log({\"point\\_scene\": point\\_scene})\n\n``` \n\n```\nwandb.log({\"protein\": wandb.Molecule(\"6lu7.pdb\")})\n\n``` \n\nLog molecular data in any of 10 file types:`pdb`, `pqr`, `mmcif`, `mcif`, `cif`, `sdf`, `sd`, `gro`, `mol2`, or `mmtf.` \n\nW&B also supports logging molecular data from SMILES strings, `rdkit` `mol` files, and `rdkit.Chem.rdchem.Mol` objects.", "matching_document_document_id": "b701ff5d-ee83-4f78-94ff-bab9439bdf4b", "query_weight": 1.2541935483870967 }, { "query_id": "4dd6d209-dba8-478c-8748-e70cb8c2ba93", "query": "logging variables in W&B", "matching_document": "Logged values don't have to be scalars. Logging any wandb object is supported.\n For example `wandb.log({\"example\": wandb.Image(\"myimage.jpg\")})` will log an\n example image which will be displayed nicely in the W&B UI.\n See the [reference documentation](https://docs.wandb.com/ref/python/data-types)\n for all of the different supported types or check out our\n [guides to logging](https://docs.wandb.ai/guides/track/log) for examples,\n from 3D molecular structures and segmentation masks to PR curves and histograms.\n `wandb.Table`s can be used to logged structured data. See our\n [guide to logging tables](https://docs.wandb.ai/guides/data-vis/log-tables)\n for details.\n\n Logging nested metrics is encouraged and is supported in the W&B UI.\n If you log with a nested dictionary like `wandb.log({\"train\":\n {\"acc\": 0.9}, \"val\": {\"acc\": 0.8}})`, the metrics will be organized into\n `train` and `val` sections in the W&B UI.", "matching_document_document_id": "f26fd629-7649-4b5c-83de-04164100d66a", "query_weight": 1.2541935483870967 }, { "query_id": "a8a88573-4e86-4b2b-a1a4-43a5870a3aab", "query": "Integrating Weights & Biases with LangChainAI, OpenAI, Huggingface Transformers", "matching_document": "## Optimize \ud83e\udd17 Hugging Face models with Weights & Biases\n\nHugging Face provides tools to quickly train neural networks for NLP (Natural Language Processing) on any task (classification, translation, question answering, etc) and any dataset with PyTorch and TensorFlow 2.0. \n\nCoupled with Weights & Biases integration, you can quickly train and monitor models for full traceability and reproducibility without any extra line of code! You just need to install the library, sign in, and your experiments will automatically be logged: \n\n```\npip install wandb\nwandb login\n\n``` \n\n**Note**: To enable logging to W&B, set `report_to` to `wandb` in your `TrainingArguments` or script. \n\nW&B integration with \ud83e\udd17 Hugging Face can automatically:\n\\* log your configuration parameters\n\\* log your losses and metrics\n\\* log gradients and parameter distributions\n\\* log your model\n\\* keep track of your code\n\\* log your system metrics (GPU, CPU, memory, temperature, etc) \n\n## Here's what the W&B interactive dashboard will look like:", "matching_document_document_id": "b42d7afe-6db5-4ce4-a388-988c7feb33e8", "query_weight": 1.2541935483870967 }, { "query_id": "b432e7ae-6584-4c53-a857-82e2f423d28c", "query": "Weights & Biases compatibility LangChainAI", "matching_document": "## Prompts Quickstart\n### Use W&B Trace with LangChain\n#### 1. Set the LANGCHAIN\\_WANDB\\_TRACING environment variable\nFirst, set the LANGCHAIN\\_WANDB\\_TRACING environment variable to true. This will turn on automated Weights & Biases logging with LangChain: \n\n```\nimport os\n\n# turn on wandb logging for langchain\nos.environ[\"LANGCHAIN\\_WANDB\\_TRACING\"] = \"true\"\n\n``` \n\nThats it! Now any call to a LangChain LLM, Chain, Tool or Agent will be logged to Weights & Biases.", "matching_document_document_id": "6adad621-230b-42f0-badc-1738fef555a6", "query_weight": 1.2541935483870967 }, { "query_id": "0c8b107d-523d-4886-9ffd-f676f422303c", "query": "versioning datasets with Weights & Biases", "matching_document": "## Dataset Versioning\n### Core Artifacts features\n1. **Upload**: Start tracking and versioning any data (files or directories) with `run.log_artifact()`. You can also track datasets in a remote filesystem (e.g. cloud storage in S3 or GCP) by reference, using a link or URI instead of the raw contents.\n2. **Version**: Define an artifact by giving it a type (`\"raw_data\"`, `\"preprocessed_data\"`, `\"balanced_data\"`) and a name (`\"imagenet_cats_10K\"`). When you log the same name again, W&B automatically creates a new version of the artifact with the latest contents.\n3. **Alias**: Set an alias like `\"best\"` or `\"production\"` to highlight the important versions in a lineage of artifacts.\n4. **Compare**: Select any two versions to browse the contents side-by-side. We're also working on a tool for dataset visualization, learn more here.\n5. **Download**: Obtain a local copy of the artifact or verify the contents by reference. \n\nFor more detail on these features, check out Artifacts how it works.", "matching_document_document_id": "7407e418-a665-438f-b605-d5dc12e4240e", "query_weight": 1.3006451612903225 }, { "query_id": "d0b61aff-ac4c-4332-853e-9efbae9f3862", "query": "Detecting dataset changes for artifact versioning in Weights & Biases", "matching_document": "## Dataset Versioning\n### Core Artifacts features\n1. **Upload**: Start tracking and versioning any data (files or directories) with `run.log_artifact()`. You can also track datasets in a remote filesystem (e.g. cloud storage in S3 or GCP) by reference, using a link or URI instead of the raw contents.\n2. **Version**: Define an artifact by giving it a type (`\"raw_data\"`, `\"preprocessed_data\"`, `\"balanced_data\"`) and a name (`\"imagenet_cats_10K\"`). When you log the same name again, W&B automatically creates a new version of the artifact with the latest contents.\n3. **Alias**: Set an alias like `\"best\"` or `\"production\"` to highlight the important versions in a lineage of artifacts.\n4. **Compare**: Select any two versions to browse the contents side-by-side. We're also working on a tool for dataset visualization, learn more here.\n5. **Download**: Obtain a local copy of the artifact or verify the contents by reference. \n\nFor more detail on these features, check out Artifacts how it works.", "matching_document_document_id": "7407e418-a665-438f-b605-d5dc12e4240e", "query_weight": 1.3006451612903225 }, { "query_id": "cd01a962-bd91-4b35-a850-9e39301f8466", "query": "How does Weights & Biases detect changes in datasets stored in S3?", "matching_document": "## How Weights and Biases Can Help with Audits & Regulatory Guidelines\n### Why use Weights and Biases?\n* The process is automated: Just with few lines of code, all models and datasets can be properly versioned and stored using W&B.\n* Minimizes chance of Error: Since there is no human in the loop, you can be sure that each client gets the same copy of the model that was saved during training (automatically).\n* Everything is reproducible/traceable: Integrating Weights and Biases in your pipelines ensures that all experiments can be reproduced. Thus, if your company is ever part of an audit, all you need to do is share the experiment ID, which will have the logs of the training script used at the time, model version served in production, model's performance scores and more! See this for an example.\n* Everything is one place: Once you integrate with Weights and Biases, all your logs, source code files, metric logs, model versions, dataset versions are in one place! This makes it really easy to have a high level view on everything even after few months/years! No longer do you need to maintain Excel spreadsheets/notebooks etc.\n* Host on your private cloud: If your data is sensitive, then Weights and Biases also provides the option to run everything locally and store model and data artifacts in your own personal buckets on AWS/GCP or any other cloud provider. Refer for more information.\n* Track everything: Weights and Biases also has the capability to track files that have been saved outside the W&B system! These are called . Reference artifacts are great for sensitive data that can be stored in your own privately managed S3 bucket, GCS bucket, HTTP file server, or even an NFS share! For an example of tracking reference files in GCP, with code and screenshots, follow our .", "matching_document_document_id": "9e10fb9a-8272-4deb-9ccc-01e961cdb37b", "query_weight": 1.3006451612903225 }, { "query_id": "5c3e2086-eccd-40ff-9f0a-3a8152a7aa1e", "query": "How does version control work in Weights & Biases for datasets?", "matching_document": "## Dataset Versioning\n### Core Artifacts features\n1. **Upload**: Start tracking and versioning any data (files or directories) with `run.log_artifact()`. You can also track datasets in a remote filesystem (e.g. cloud storage in S3 or GCP) by reference, using a link or URI instead of the raw contents.\n2. **Version**: Define an artifact by giving it a type (`\"raw_data\"`, `\"preprocessed_data\"`, `\"balanced_data\"`) and a name (`\"imagenet_cats_10K\"`). When you log the same name again, W&B automatically creates a new version of the artifact with the latest contents.\n3. **Alias**: Set an alias like `\"best\"` or `\"production\"` to highlight the important versions in a lineage of artifacts.\n4. **Compare**: Select any two versions to browse the contents side-by-side. We're also working on a tool for dataset visualization, learn more here.\n5. **Download**: Obtain a local copy of the artifact or verify the contents by reference. \n\nFor more detail on these features, check out Artifacts how it works.", "matching_document_document_id": "7407e418-a665-438f-b605-d5dc12e4240e", "query_weight": 1.3006451612903225 }, { "query_id": "235220d7-5ae2-4f47-8032-b3dc3dda1bb1", "query": "Weights & Biases run organization for hyperparameter tuning", "matching_document": "## Run\n| Attributes | |\n| --- | --- |\n| `summary` | (Summary) Single values set for each `wandb.log()` key. By default, summary is set to the last value logged. You can manually set summary to the best value, like max accuracy, instead of the final value. |\n| `config` | Config object associated with this run. |\n| `dir` | The directory where files associated with the run are saved. |\n| `entity` | The name of the W&B entity associated with the run. Entity can be a username or the name of a team or organization. |\n| `group` | Name of the group associated with the run. Setting a group helps the W&B UI organize runs in a sensible way. If you are doing a distributed training you should give all of the runs in the training the same group. If you are doing cross-validation you should give all the cross-validation folds the same group. |\n| `id` | Identifier for this run. |\n| `mode` | For compatibility with `0.9.x` and earlier, deprecate eventually. |\n| `name` | Display name of the run. Display names are not guaranteed to be unique and may be descriptive. By default, they are randomly generated. |\n| `notes` | Notes associated with the run, if there are any. Notes can be a multiline string and can also use markdown and latex equations inside `$$`, like `$x + 3$`. |\n| `path` | Path to the run. Run paths include entity, project, and run ID, in the format `entity/project/run_id`. |\n| `project` | Name of the W&B project associated with the run. |\n| `resumed` | True if the run was resumed, False otherwise. |\n| `settings` | A frozen copy of run's Settings object. |\n| `start_time` | Unix timestamp (in seconds) of when the run started. |\n| `starting_step` | The first step of the run. |\n| `step` | Current value of the step. This counter is incremented by `wandb.log`. |\n| `sweep_id` | ID of the sweep associated with the run, if there is one. |", "matching_document_document_id": "cdf82567-6205-4b8c-a0e7-ee2333c5bcde", "query_weight": 1.3470967741935482 }, { "query_id": "e9e936bc-0e65-4d62-91ef-e59b6e470abc", "query": "averaging scores across CV sets in wandb", "matching_document": "## Setup\n#### Step 1: Write get_score function\nThe n_estimators parameter supplied to get_score() is used when setting the number of trees in the random forest model. \n\nWe'll use wandb to log the metrics, which can later be used to compare the performance of different models. \n\n```\nimport wandb\ndef get_score(n_estimators):\n\"\"\"\nReturn the average MAE over 3 CV folds of random forest model.\n\"\"\"\nwandb.init(project=\"Kaggle-ML-CV\",name=str(n_estimators)+'_estimators')\nmy_pipeline = Pipeline(steps=[\n('preprocessor', SimpleImputer()),\n('model', RandomForestRegressor(n_estimators, random_state=0))\n])\nscores = -1 * cross_val_score(my_pipeline, X, y,\ncv=3,\nscoring='neg_mean_absolute_error')\nwandb.log({\"Mean score\":scores.mean()})\nreturn scores.mean()\n```", "matching_document_document_id": "56710c7c-5c49-40a9-bbae-06e1bf5dbd90", "query_weight": 1.3470967741935482 }, { "query_id": "af7fdb72-5d91-40da-a253-bad3275c95e1", "query": "How to use WandB's run naming engine for grouped runs?", "matching_document": "## Group Runs\n\nGroup W&B Runs \n\nGroup individual jobs into experiments by passing a unique **group** name to **wandb.init()**. \n\n## Use Cases \n\n1. **Distributed training:** Use grouping if your experiments are split up into different pieces with separate training and evaluation scripts that should be viewed as parts of a larger whole.\n2. **Multiple processes**: Group multiple smaller processes together into an experiment.\n3. **K-fold cross-validation**: Group together runs with different random seeds to see a larger experiment. Here's an example of k-fold cross-validation with sweeps and grouping. \n\nThere are three ways to set grouping: \n\n### 1. Set group in your script \n\nPass an optional group and job\\_type to wandb.init(). This gives you a dedicated group page for each experiment, which contains the individual runs. For example:`wandb.init(group=\"experiment_1\", job_type=\"eval\")` \n\n### 2. Set a group environment variable \n\n### 3. Toggle grouping in the UI \n\n## Distributed training with grouping", "matching_document_document_id": "fe624bd2-6bf7-487a-8e2d-5a4ca406b592", "query_weight": 1.3470967741935482 }, { "query_id": "9b27a638-29b5-4f23-a686-69568ade1235", "query": "How to troubleshoot artifact download issues in Weights & Biases?", "matching_document": "## Download and use artifacts\n#### Download and use an artifact stored on W&B\nUse the object returned to download all the contents of the artifact: \n\n```\ndatadir = artifact.download()\n\n``` \n\nYou can optionally pass a path to the root parameter to download the contents of the artifact to a specific directory. For more information, see the Python SDK Reference Guide. \n\nUse the `get_path` method to download only subset of files: \n\n```\npath = artifact.get\\_path(name)\n\n``` \n\nThis fetches only the file at the path `name`. It returns an `Entry` object with the following methods: \n\n* `Entry.download`: Downloads file from the artifact at path `name`\n* `Entry.ref`: If the entry was stored as a reference using `add_reference`, returns the URI \n\nReferences that have schemes that W&B knows how to handle can be downloaded just like artifact files. For more information, see Track external files. \n\nFirst, import the W&B SDK. Next, create an artifact from the Public API Class. Provide the entity, project, artifact, and alias associated with that artifact:", "matching_document_document_id": "a42e4236-eca7-4763-a693-b44e8187ca6f", "query_weight": 1.1148387096774193 }, { "query_id": "f2c19e7b-6368-4102-92f0-29e3b5612ba3", "query": "Examples of accessing Weights & Biases artifact metadata programmatically", "matching_document": "##### Examples:\n| Attributes | |\n| --- | --- |\n| `aliases` | The aliases associated with this artifact. The list is mutable and calling `save()` will persist all alias changes. |\n| `collection` | The collection this artifact was retrieved from. If this artifact was retrieved from a portfolio / linked collection, that collection will be returned rather than the source sequence. |\n| `commit_hash` | The hash returned when this artifact was committed. |\n| `created_at` | The time at which the artifact was created. |\n| `description` | The artifact description. Free text that offers a user-set description of the artifact. |\n| `digest` | The logical digest of the artifact. The digest is the checksum of the artifact's contents. If an artifact has the same digest as the current `latest` version, then `log_artifact` is a no-op. |\n| `entity` | The name of the entity of the secondary (portfolio) artifact collection. |\n| `file_count` | The number of files (including references). |\n| `id` | The artifact's ID. |\n| `manifest` | The artifact's manifest. The manifest lists all of its contents, and can't be changed once the artifact has been logged. |\n| `metadata` | User-defined artifact metadata. Structured data associated with the artifact. |\n| `name` | The artifact name and version in its secondary (portfolio) collection. A string with the format {collection}:{alias}. Before the artifact is saved, contains only the name since the version is not yet known. |\n| `project` | The name of the project of the secondary (portfolio) artifact collection. |\n| `qualified_name` | The entity/project/name of the secondary (portfolio) collection. |\n| `size` | The total size of the artifact in bytes. Includes any references tracked by this artifact. |\n| `source_collection` | The artifact's primary (sequence) collection. |\n| `source_entity` | The name of the entity of the primary (sequence) artifact collection. |", "matching_document_document_id": "9a40a145-892c-4888-999e-cd65b7b3d0ef", "query_weight": 1.1148387096774193 }, { "query_id": "9c1bacf9-6592-44b1-9c06-b2db78b32026", "query": "How to find the storage location of an artifact in Weights & Biases?", "matching_document": "## Artifact\n### Methods\n#### `get_added_local_path_name`\n\nView source \n\n```\nget\\_added\\_local\\_path\\_name(\nlocal\\_path: str\n) -> Optional[str]\n\n``` \n\nGet the artifact relative name of a file added by a local filesystem path. \n\n| Arguments | |\n| --- | --- |\n| `local_path` | The local path to resolve into an artifact relative name. | \n\n| Returns | |\n| --- | --- |\n| The artifact relative name. | | \n\n#### Examples: \n\nBasic usage: \n\n```\nartifact = wandb.Artifact(\"my\\_dataset\", type=\"dataset\")\nartifact.add\\_file(\"path/to/file.txt\", name=\"artifact/path/file.txt\")\n\n# Returns `artifact/path/file.txt`:\nname = artifact.get\\_added\\_local\\_path\\_name(\"path/to/file.txt\")\n\n```", "matching_document_document_id": "9e0404bc-42ab-435c-80b4-d5f5b1d553e3", "query_weight": 1.1148387096774193 }, { "query_id": "238c9f7e-4113-471c-b7d9-0e72c3122641", "query": "Best practices for troubleshooting wandb service start issues.", "matching_document": "## Troubleshoot Sweeps\n\nTroubleshoot W&B Sweeps \n\nTroubleshoot common error messages with the guidance suggested. \n\n### `CommError, Run does not exist` and `ERROR Error uploading` \n\nYour W&B Run ID might be defined if these two error messages are both returned. As an example, you might have a similar code snippet defined somewhere in your Jupyter Notebooks or Python script: \n\n```\nwandb.init(id=\"some-string\")\n\n``` \n\nYou can not set a Run ID for W&B Sweeps because W&B automatically generates random, unique IDs for Runs created by W&B Sweeps. \n\nW&B Run IDs need to be unique within a project. \n\nWe recommend you pass a name to the name parameter when you initialized W&B, if you want to set a custom name that will appear on tables and graphs. For example: \n\n```\nwandb.init(name=\"a helpful readable run name\")\n\n``` \n\n### `Cuda out of memory` \n\n### `anaconda 400 error` \n\nThe following error usually occurs when you do not log the metric that you are optimizing:", "matching_document_document_id": "13152c97-59c9-456d-b955-55a590204565", "query_weight": 1.7651612903225806 }, { "query_id": "6a1562de-2c20-4342-a1c0-3311138035ea", "query": "Common wandb setup errors and how to solve them", "matching_document": "## init\n#### Add metadata about the run to the config\n\nPass a dictionary-style object as the `config` keyword argument to add\nmetadata, like hyperparameters, to your run. \n\n```\nimport wandb\n\nconfig = {\"lr\": 3e-4, \"batch\\_size\": 32}\nconfig.update({\"architecture\": \"resnet\", \"depth\": 34})\nwandb.init(config=config)\n\n``` \n\n| Raises | |\n| --- | --- |\n| `Error` | if some unknown or internal error happened during the run initialization. |\n| `AuthenticationError` | if the user failed to provide valid credentials. |\n| `CommError` | if there was a problem communicating with the WandB server. |\n| `UsageError` | if the user provided invalid arguments. |\n| `KeyboardInterrupt` | if user interrupts the run. | \n\n| Returns | |\n| --- | --- |\n| A `Run` object. | |", "matching_document_document_id": "1974561f-5b98-4306-91a8-59f948714a10", "query_weight": 1.7651612903225806 }, { "query_id": "f765a5d8-6a50-48d0-8e02-751a175875f6", "query": "What causes 'Error communicating with wandb process'?", "matching_document": "## Experiments FAQ\n\nFrequently Asked Questions About Experiments \n\nThe proceeding questions are commonly asked questions about W&B Artifacts. \n\n### How do I launch multiple runs from one script? \n\nUse `wandb.init` and `run.finish()` to log multiple Runs from one script: \n\n1. `run = wandb.init(reinit=True)`: Use this setting to allow reinitializing runs\n2. `run.finish()`: Use this at the end of your run to finish logging for that run \n\n```\nimport wandb\n\nfor x in range(10):\nrun = wandb.init(reinit=True)\nfor y in range(100):\nwandb.log({\"metric\": x + y})\nrun.finish()\n\n``` \n\nAlternatively you can use a python context manager which will automatically finish logging: \n\n```\nimport wandb\n\nfor x in range(10):\nrun = wandb.init(reinit=True)\nwith run:\nfor y in range(100):\nrun.log({\"metric\": x + y})\n\n``` \n\n### `InitStartError: Error communicating with wandb process` \n\nThis error indicates that the library is having difficulty launching the process which synchronizes data to the server.", "matching_document_document_id": "9b1af9fa-cbac-449f-a522-8d9199665bcf", "query_weight": 1.7651612903225806 }, { "query_id": "87ed8282-6b4f-4f33-920b-64bfcf3dab81", "query": "debug wandb startup error", "matching_document": "if run_result is None:\n error_message = (\n f\"Run initialization has timed out after {timeout} sec. \"\n f\"\\nPlease refer to the documentation for additional information: {wburls.get('doc_start_err')}\"\n )\n # We're not certain whether the error we encountered is due to an issue\n # with the server (a \"CommError\") or if it's a problem within the SDK (an \"Error\").\n # This means that the error could be a result of the server being unresponsive,\n # or it could be because we were unable to communicate with the wandb service.\n error = CommError(error_message)\n run_init_handle._cancel()\n elif run_result.HasField(\"error\"):\n error = ProtobufErrorHandler.to_exception(run_result.error)\n\n if error is not None:\n logger.error(f\"encountered error: {error}\")\n if not manager:\n # Shutdown the backend and get rid of the logger\n # we don't need to do console cleanup at this point\n backend.cleanup()\n self.teardown()\n raise error", "matching_document_document_id": "6a82b1c5-eb69-4bbd-8c90-295ca3c5021f", "query_weight": 1.7651612903225806 }, { "query_id": "99c87395-8f7d-437f-82b5-f9330f7cabb7", "query": "Efficient data handling with W&B and Pandas for machine learning projects", "matching_document": "## Code\n### 1: Data Artifacts with W&B and Pandas Profiling\n#### Pandas Profiling\n```\n## Common data splitting function with some hard coded parameters\n# We use this to split the data into training and \"production\" data for our evaluation later\ndef sample_split_data(dataset, seed=1):\ndata = dataset.sample(frac=0.95, random_state=seed)\ndata_unseen = dataset.drop(data.index)\n\ndata.reset_index(inplace=True, drop=True)\ndata_unseen.reset_index(inplace=True, drop=True)\n\nprint('Data for Modeling: ' + str(data.shape))\nprint('Unseen Data For Predictions ' + str(data_unseen.shape))\n\nreturn (data, data_unseen)\n\n``` \n\nIn our example we will be finding ourselves utilizing our data pool for 3 general categories of data batches we may interact with as an ML practictioner:\n- Training Data\n- Validation Data\n- \"Production\" Unseen Data \n\nIn our case our package of choice `pycaret` will be handling the partitioning of data into training and validation from a pool of provided data (the data for modeling)", "matching_document_document_id": "83095c61-c601-448b-aed5-8b755ca4c849", "query_weight": 1.2541935483870967 }, { "query_id": "e89cc94e-ad37-47ee-a6da-5e53203a6900", "query": "loop through W&B run tables", "matching_document": "## Runs\n### View all Runs in a Project\nView Runs associated to a project with the W&B App UI. Navigate to the W&B App and search for the name of your project. \n\nIn the following example we search for a project called \"my-first-run\": \n\nSelect the project. This will redirect you to that project's Workspace. A Project's Workspace gives you a personal sandbox to compare runs. Use projects to organize models that can be compared, working on the same problem with different architectures, hyperparameters, datasets, preprocessing and so on. \n\nWithin your project's workspace, you will see a table labeled **Runs**. This table lists all the Runs that are in your project. In other words, these runs were provided a `project` argument when it was created. \n\nThe following image demonstrates a project workspace called \"sweep-demo\": \n\nThe **Runs Sidebar** lists of all the runs in your project. Hover your mouse over a single Run to modify or view the following:", "matching_document_document_id": "84f9be37-ed17-492a-ac77-3b352e9134e5", "query_weight": 1.2541935483870967 }, { "query_id": "58565320-ffb6-49d8-9f44-500f1b5959d0", "query": "Automating data download from W&B to Pandas DataFrame", "matching_document": "## Export Table Data\n\nLike all W&B Artifacts, Tables can be converted into pandas dataframes for easy data exporting. \n\n## Convert `table` to `artifact` \n\nFirst, you'll need to convert the table to an artifact. The easiest way to do this using `artifact.get(table, \"table_name\")`: \n\n```\n# Create and log a new table.\nwith wandb.init() as r:\nartifact = wandb.Artifact(\"my\\_dataset\", type=\"dataset\")\ntable = wandb.Table(\ncolumns=[\"a\", \"b\", \"c\"], data=[(i, i \\* 2, 2\\*\\*i) for i in range(10)]\n)\nartifact.add(table, \"my\\_table\")\nwandb.log\\_artifact(artifact)\n\n# Retrieve the created table using the artifact you created.\nwith wandb.init() as r:\nartifact = r.use\\_artifact(\"my\\_dataset:latest\")\ntable = artifact.get(\"my\\_table\")\n\n``` \n\n## Convert `artifact` to Dataframe \n\nThen, convert the table into a dataframe: \n\n```\n# Following from the last code example:\ndf = table.get\\_dataframe()\n\n``` \n\n## Export Data \n\nNow you can export using any method dataframe supports:", "matching_document_document_id": "742d370f-04d6-468b-8dfb-2b241cb96084", "query_weight": 1.2541935483870967 }, { "query_id": "48e7c951-f8a2-4efc-ad9a-4d9da48d0385", "query": "What are the best practices for managing W&B API keys in a cloud environment?", "matching_document": "## Environment Variables\n\nW&B Environment Variables \n\nWhen you're running a script in an automated environment, you can control **wandb** with environment variables set before the script runs or within the script. \n\n```\n# This is secret and shouldn't be checked into version control\nWANDB\\_API\\_KEY=$YOUR\\_API\\_KEY\n# Name and notes optional\nWANDB\\_NAME=\"My first run\"\nWANDB\\_NOTES=\"Smaller learning rate, more regularization.\"\n\n``` \n\n```\n# Only needed if you don't check in the wandb/settings file\nWANDB\\_ENTITY=$username\nWANDB\\_PROJECT=$project\n\n``` \n\n```\n# If you don't want your script to sync to the cloud\nos.environ[\"WANDB\\_MODE\"] = \"offline\"\n\n``` \n\n## Optional Environment Variables \n\nUse these optional environment variables to do things like set up authentication on remote machines. \n\n## Singularity Environments \n\n## Running on AWS \n\n## Common Questions \n\n### Automated runs and service accounts \n\n### Do environment variables overwrite the parameters passed to wandb.init()? \n\n### Turn off logging", "matching_document_document_id": "ee47914b-f05e-4b88-885c-74b51e9fb82e", "query_weight": 1.2541935483870967 }, { "query_id": "c0b6df83-9774-40b6-99ac-2b3903598bfd", "query": "Automating W&B authentication on cloud servers", "matching_document": "## Trigger CI/CD events with artifact changes\n### Create a webhook automation\nAutomate a webhook based on an action with the W&B App UI. To do this, you will first establish a webhook, then you will configure the webhook automation. \n\n### Add a secret for authentication or authorization \n\nSecrets are team-level variables that let you obfuscate private strings such as credentials, API keys, passwords, tokens, and more. W&B recommends you use secrets to store any string that you want to protect the plain text content of. \n\nTo use a secret in your webhook, you must first add that secret to your team's secret manager. \n\n:::info\n\\* Only W&B Admins can create, edit, or delete a secret.\n\\* Skip this section if the external server you send HTTP POST requests to does not use secrets.\n\\* Secrets are also available if you use W&B Server in an Azure, GCP, or AWS deployment. Connect with your W&B account team to discuss how you can use secrets in W&B if you use a different deployment type.\n::: \n\nThere are two types of secrets W&B suggests that you create when you use a webhook automation:", "matching_document_document_id": "851ff26a-c8b7-4e99-9c95-a5e498fbbbf1", "query_weight": 1.2541935483870967 }, { "query_id": "4570c6f4-d308-4af8-821e-f3039b1db930", "query": "Access W&B model without authentication", "matching_document": "## Anonymous Mode\n#### How does someone without an account see results?\n1. **Auto-create temporary account:** W&B checks for an account that's already signed in. If there's no account, we automatically create a new anonymous account and save that API key for the session.\n2. **Log results quickly:** The user can run and re-run the script, and automatically see results show up in the W&B dashboard UI. These unclaimed anonymous runs will be available for 7 days.\n3. **Claim data when it's useful**: Once the user finds valuable results in W&B, they can easily click a button in the banner at the top of the page to save their run data to a real account. If they don't claim a run, it will be deleted after 7 days. \n\n:::caution\n**Anonymous run links are sensitive**. These links allow anyone to view and claim the results of an experiment for 7 days, so make sure to only share links with people you trust. If you're trying to share results publicly, but hide the author's identity, please contact us at support@wandb.com to share more about your use case.\n:::", "matching_document_document_id": "9aa538cf-da7d-4859-b987-1c2503186a6c", "query_weight": 1.2541935483870967 }, { "query_id": "7ae682a3-2c43-4f04-984e-27e4934c6a26", "query": "What are the best practices for managing W&B authentication keys on cloud servers?", "matching_document": "## Triggering CI/CD events with model registry changes\n### Create a webhook automation\n#### Add a secret for authentication or authorization\nYou are responsible for configuring security measures that satisfy your security needs. \n\nW&B strongly recommends that you store secrets in a W&B instance of a cloud secrets manager provided by AWS, GCP, or Azure. Secret managers provided by AWS, GCP, and Azure are configured with advanced security capabilities. \n\nW&B does not recommend that you use a Kubernetes cluster as the backend of your secrets store. Consider a Kubernetes cluster only if you are not able to use a W&B instance of a cloud secrets manager (AWS, GCP, or Azure), and you understand how to prevent security vulnerabilities that can occur if you use a cluster.\n:::", "matching_document_document_id": "6566fd6b-6c21-42c1-854a-3e411ccb7091", "query_weight": 1.2541935483870967 }, { "query_id": "6fb3de88-eb8f-4c0d-8911-b9009a9e4d97", "query": "recent changes in WandB UI features", "matching_document": "## W&B Launch\n* Error messages: Run queue items now have associated errors with them, viewable right within the W&B UI. This should make debugging runs with errors less painful, and further optimize and improve the efficiency of GPU usage\n* Queue selector: Non-active queues will now be automatically filtered out. Having non-active queues previously clogging up the selector made it unwieldy\u2014especially for large teams\u2014to scroll looking for the right queue to use.\n* Create job in CLI: Using wandb job create users can now create a job in the CLI from git code or image source. This removes one previously necessary step in running a job.\n* Support for AKS - Launch now supports Azure Kubernetes Service! If you\u02bcre an AKS user, please reach out to your dedicated CSM and we\u02bcll be happy to help you get set up. \n\n---\nThat's all for this month's edition of our product newsletter. Here's a bit more on recent improvements, partnerships, and further detail about how to use some of our newest features: \n\n\"", "matching_document_document_id": "85a51e9f-cb8a-4138-9386-3501754edd59", "query_weight": 0.9290322580645161 }, { "query_id": "34e406d8-73c0-4025-a14a-5708e9eada9c", "query": "Why is Wandb not loading the specified YAML configuration file?", "matching_document": "def load_sweep_config(sweep_config_path: str) -> Optional[Dict[str, Any]]:\n \"\"\"Load a sweep yaml from path.\"\"\"\n try:\n yaml_file = open(sweep_config_path)\n except OSError:\n wandb.termerror(f\"Couldn't open sweep file: {sweep_config_path}\")\n return None\n try:\n config: Optional[Dict[str, Any]] = yaml.safe_load(yaml_file)\n except yaml.YAMLError as err:\n wandb.termerror(f\"Error in configuration file: {err}\")\n return None\n if not config:\n wandb.termerror(\"Configuration file is empty\")\n return None\n return config", "matching_document_document_id": "288bcbe1-0e07-4244-bd5c-50550e7c1e2b", "query_weight": 0.8825806451612903 }, { "query_id": "ba753845-3cd7-492e-afb0-9940611aaac2", "query": "What are common issues with Wandb configuration file loading?", "matching_document": "def load_sweep_config(sweep_config_path: str) -> Optional[Dict[str, Any]]:\n \"\"\"Load a sweep yaml from path.\"\"\"\n try:\n yaml_file = open(sweep_config_path)\n except OSError:\n wandb.termerror(f\"Couldn't open sweep file: {sweep_config_path}\")\n return None\n try:\n config: Optional[Dict[str, Any]] = yaml.safe_load(yaml_file)\n except yaml.YAMLError as err:\n wandb.termerror(f\"Error in configuration file: {err}\")\n return None\n if not config:\n wandb.termerror(\"Configuration file is empty\")\n return None\n return config", "matching_document_document_id": "288bcbe1-0e07-4244-bd5c-50550e7c1e2b", "query_weight": 0.8825806451612903 }, { "query_id": "b443b423-3b05-4d18-a12f-41a380f8025e", "query": "Wandb config file not loading", "matching_document": "def load_sweep_config(sweep_config_path: str) -> Optional[Dict[str, Any]]:\n \"\"\"Load a sweep yaml from path.\"\"\"\n try:\n yaml_file = open(sweep_config_path)\n except OSError:\n wandb.termerror(f\"Couldn't open sweep file: {sweep_config_path}\")\n return None\n try:\n config: Optional[Dict[str, Any]] = yaml.safe_load(yaml_file)\n except yaml.YAMLError as err:\n wandb.termerror(f\"Error in configuration file: {err}\")\n return None\n if not config:\n wandb.termerror(\"Configuration file is empty\")\n return None\n return config", "matching_document_document_id": "288bcbe1-0e07-4244-bd5c-50550e7c1e2b", "query_weight": 0.8825806451612903 }, { "query_id": "bc4f527c-1303-416f-987c-c73137f5fdde", "query": "Wandb configuration file loading issue", "matching_document": "def load_sweep_config(sweep_config_path: str) -> Optional[Dict[str, Any]]:\n \"\"\"Load a sweep yaml from path.\"\"\"\n try:\n yaml_file = open(sweep_config_path)\n except OSError:\n wandb.termerror(f\"Couldn't open sweep file: {sweep_config_path}\")\n return None\n try:\n config: Optional[Dict[str, Any]] = yaml.safe_load(yaml_file)\n except yaml.YAMLError as err:\n wandb.termerror(f\"Error in configuration file: {err}\")\n return None\n if not config:\n wandb.termerror(\"Configuration file is empty\")\n return None\n return config", "matching_document_document_id": "288bcbe1-0e07-4244-bd5c-50550e7c1e2b", "query_weight": 0.8825806451612903 }, { "query_id": "cbaf1eff-f4e9-4593-b5a5-d6f4811939cb", "query": "Troubleshooting Wandb --configs flag not working", "matching_document": "## Configure Experiments\n### File-Based Configs\nKey-value pairs are automatically passed to `wandb.config` if you create a file called `config-defaults.yaml`. \n\nThe proceeding code snippet demonstrates a sample `config-defaults.yaml` YAML file: \n\n```\n# config-defaults.yaml\n# sample config defaults file\nepochs:\ndesc: Number of epochs to train over\nvalue: 100\nbatch\\_size:\ndesc: Size of each mini-batch\nvalue: 32\n\n``` \n\nYou can overwrite automatically passed by a `config-defaults.yaml`. To do so , pass values to the `config` argument of `wandb.init`. \n\nYou can also load different config files with the command line argument `--configs`. \n\n### Example use case for file-based configs \n\nSuppose you have a YAML file with some metadata for the run, and then a dictionary of hyperparameters in your Python script. You can save both in the nested `config` object:", "matching_document_document_id": "93f4f4c2-1aa0-4d0b-b7e8-171f2df8b8f3", "query_weight": 0.8825806451612903 }, { "query_id": "102f77e4-b375-4629-ba4d-c90d32edac67", "query": "Impact of run crashes on wandb.init resume functionality", "matching_document": "## GCP with Weights & Biases\n### Advanced Features: Resuming and Grouping Runs\n\n### Resuming Runs \n\nIn machine learning, MemoryErrors are extremely common - even when training is taking place on GPUs. When these errors occur, model training is impacted. In addition to MemoryErrors, it is also common for those new to Machine Learning to encounter a power failure and lose the progress of days of training. This is all to say that setting up your development environment thoughtfully will pay dividends in the long run. \n\nBut let\u2019s say you are training a model, water spills on your computer, and your computer shuts down unexpectedly. Weights and Biases help you prepare for these types of situations by allowing users to resume a run if it was not completed. Let\u2019s use an example to demonstrate. \n\nFirst things first - you need to set the resume argument in wandb.init() to True: \n\n```\nwandb.init(project=\"resume-read-group-runs\", name=\"resume_runs\", resume=True)\n``` \n\n### Grouping Runs", "matching_document_document_id": "7604511c-d389-4b7d-83f8-b4b836751300", "query_weight": 1.0219354838709678 }, { "query_id": "3663db5d-899d-41f5-89ca-d35fdb8677ec", "query": "difference between allow and auto in wandb", "matching_document": "metadata will be stored. When you call `download()` on an artifact,\n this is the directory where downloaded files will be saved. By default,\n this is the `./wandb` directory.\n resume: (bool, str, optional) Sets the resuming behavior. Options:\n `\"allow\"`, `\"must\"`, `\"never\"`, `\"auto\"` or `None`. Defaults to `None`.\n Cases:\n - `None` (default): If the new run has the same ID as a previous run,\n this run overwrites that data.\n - `\"auto\"` (or `True`): if the previous run on this machine crashed,\n automatically resume it. Otherwise, start a new run.\n - `\"allow\"`: if id is set with `init(id=\"UNIQUE_ID\")` or\n `WANDB_RUN_ID=\"UNIQUE_ID\"` and it is identical to a previous run,\n wandb will automatically resume the run with that id. Otherwise,\n wandb will start a new run.\n - `\"never\"`: if id is set with `init(id=\"UNIQUE_ID\")` or\n `WANDB_RUN_ID=\"UNIQUE_ID\"` and it is identical to a previous run,\n wandb will crash.", "matching_document_document_id": "bc7be183-f52f-4562-a551-f2113d440deb", "query_weight": 1.0219354838709678 }, { "query_id": "03d32819-3195-44e5-907f-392339622dcc", "query": "run resuming", "matching_document": "## Resume Runs\n#### Resume Guidance\n1. As described above, runs can be resumed by passing`resume=True` to `wandb.init()`. This can be thought of as auto-resuming, where we \u201cautomatically\u201d pick up from where an aborted run left off. If your process doesn't exit successfully, the next time you run it W&B will start logging from the last step.\n2. Note: This only works if you are running your script in the same directory as the one that failed as the file is stored at: `wandb/wandb-resume.json`.\n3. The other form of resume requires you to provide the actual run id: `wandb.init(id=run_id)` and then when you resume (if you want to be sure that it is resuming, you do `wandb.init(id=run_id, resume=\"must\")`.\n4. You can also have full control over resuming if you manage the `run_id`. We provide a utility to generate `run_id`: `wandb.util.generate_id()`. As long as you set the id to one of these unique ids for each unique run, you can say `resume=\"allow\"` and W&B will automatically resume the run with that id.More context regarding automatic and controlled resuming can be found in this section.\n2. `wandb.restore`\n\\* This will allow you to log new historical values for your metrics to a run starting from where you left off but does not take care of re-establishing the state of your code, you will need to make sure you have written checkpoints that you can load!\n\\* You can use `wandb.save` to record the state of your run via checkpoint files. Create a checkpoint file through `wandb.save()`, which can then be used through `wandb.init(resume=)`. This report illustrates how to save and restore models with W&B.", "matching_document_document_id": "3effe167-849e-453c-a198-3e826eb7e84a", "query_weight": 1.0219354838709678 }, { "query_id": "bbfc6958-6cd2-4e9c-8f3b-4ef642771752", "query": "Run ID matching logic in wandb for resume parameter", "matching_document": "| **WANDB\\_RESUME** | By default this is set to *never*. If set to *auto* wandb will automatically resume failed runs. If set to *must* forces the run to exist on startup. If you want to always generate your own unique ids, set this to *allow* and always set **WANDB\\_RUN\\_ID**. |\n| **WANDB\\_RUN\\_GROUP** | Specify the experiment name to automatically group runs together. See grouping for more info. |\n| **WANDB\\_RUN\\_ID** | Set this to a globally unique string (per project) corresponding to a single run of your script. It must be no longer than 64 characters. All non-word characters will be converted to dashes. This can be used to resume an existing run in cases of failure. |\n| **WANDB\\_SILENT** | Set this to **true** to silence wandb log statements. If this is set all logs will be written to **WANDB\\_DIR**/debug.log |\n| **WANDB\\_SHOW\\_RUN** | Set this to **true** to automatically open a browser with the run url if your operating system supports it. |\n| **WANDB\\_TAGS** | A comma separated list of tags to be applied to the run. |\n| **WANDB\\_USERNAME** | The username of a member of your team associated with the run. This can be used along with a service account API key to enable attribution of automated runs to members of your team. |\n| **WANDB\\_USER\\_EMAIL** | The email of a member of your team associated with the run. This can be used along with a service account API key to enable attribution of automated runs to members of your team. |", "matching_document_document_id": "f987f789-7ae7-4e30-85ca-f2ead4d1a892", "query_weight": 1.0219354838709678 }, { "query_id": "af4a846a-174b-4628-907c-ddc1b872e5b0", "query": "sweep plan schema", "matching_document": "## Sweep configuration structure\n### Basic structure\ntrain.py\n```\nsweep\\_configuration = {\n\"name\": \"sweepdemo\",\n\"method\": \"bayes\",\n\"metric\": {\"goal\": \"minimize\", \"name\": \"validation\\_loss\"},\n\"parameters\": {\n\"learning\\_rate\": {\"min\": 0.0001, \"max\": 0.1},\n\"batch\\_size\": {\"values\": [16, 32, 64]},\n\"epochs\": {\"values\": [5, 10, 15]},\n\"optimizer\": {\"values\": [\"adam\", \"sgd\"]},\n},\n}\n\n``` \n\nDefine a sweep configuration in a YAML file if you want to manage sweeps interactively from the command line (CLI) \n\nconfig.yaml\n```\nprogram: train.py\nname: sweepdemo\nmethod: bayes\nmetric:\ngoal: minimize\nname: validation\\_loss\nparameters:\nlearning\\_rate:\nmin: 0.0001\nmax: 0.1\nbatch\\_size:\nvalues: [16, 32, 64]\nepochs:\nvalues: [5, 10, 15]\noptimizer:\nvalues: [\"adam\", \"sgd\"]\n\n```", "matching_document_document_id": "2a1ce862-7280-49c1-80c4-e4dfdd16914e", "query_weight": 0.7896774193548387 }, { "query_id": "a1b5b73f-cbf8-4a67-883f-ad9a462d2418", "query": "YAML JSON sweep configuration", "matching_document": "## Sweep configuration structure\n### Basic structure\ntrain.py\n```\nsweep\\_configuration = {\n\"name\": \"sweepdemo\",\n\"method\": \"bayes\",\n\"metric\": {\"goal\": \"minimize\", \"name\": \"validation\\_loss\"},\n\"parameters\": {\n\"learning\\_rate\": {\"min\": 0.0001, \"max\": 0.1},\n\"batch\\_size\": {\"values\": [16, 32, 64]},\n\"epochs\": {\"values\": [5, 10, 15]},\n\"optimizer\": {\"values\": [\"adam\", \"sgd\"]},\n},\n}\n\n``` \n\nDefine a sweep configuration in a YAML file if you want to manage sweeps interactively from the command line (CLI) \n\nconfig.yaml\n```\nprogram: train.py\nname: sweepdemo\nmethod: bayes\nmetric:\ngoal: minimize\nname: validation\\_loss\nparameters:\nlearning\\_rate:\nmin: 0.0001\nmax: 0.1\nbatch\\_size:\nvalues: [16, 32, 64]\nepochs:\nvalues: [5, 10, 15]\noptimizer:\nvalues: [\"adam\", \"sgd\"]\n\n```", "matching_document_document_id": "2a1ce862-7280-49c1-80c4-e4dfdd16914e", "query_weight": 0.7896774193548387 }, { "query_id": "f2b35ed2-eddb-44ec-80b1-620db172d670", "query": "JSON format for W&B sweep plan", "matching_document": "## Sweep configuration structure\n### Basic structure\ntrain.py\n```\nsweep\\_configuration = {\n\"name\": \"sweepdemo\",\n\"method\": \"bayes\",\n\"metric\": {\"goal\": \"minimize\", \"name\": \"validation\\_loss\"},\n\"parameters\": {\n\"learning\\_rate\": {\"min\": 0.0001, \"max\": 0.1},\n\"batch\\_size\": {\"values\": [16, 32, 64]},\n\"epochs\": {\"values\": [5, 10, 15]},\n\"optimizer\": {\"values\": [\"adam\", \"sgd\"]},\n},\n}\n\n``` \n\nDefine a sweep configuration in a YAML file if you want to manage sweeps interactively from the command line (CLI) \n\nconfig.yaml\n```\nprogram: train.py\nname: sweepdemo\nmethod: bayes\nmetric:\ngoal: minimize\nname: validation\\_loss\nparameters:\nlearning\\_rate:\nmin: 0.0001\nmax: 0.1\nbatch\\_size:\nvalues: [16, 32, 64]\nepochs:\nvalues: [5, 10, 15]\noptimizer:\nvalues: [\"adam\", \"sgd\"]\n\n```", "matching_document_document_id": "2a1ce862-7280-49c1-80c4-e4dfdd16914e", "query_weight": 0.7896774193548387 }, { "query_id": "b4e834b6-6707-4305-aa27-c2dcef2039ac", "query": "sweep configuration without YAML", "matching_document": "## Sweep configuration structure\n\nDefine sweep configuration for hyperparameter tuning. \n\nA W&B Sweep combines a strategy for exploring hyperparameter values with the code that evaluates them. The strategy can be as simple as trying every option or as complex as Bayesian Optimization and Hyperband (BOHB). \n\nDefine a sweep configuration either in a Python dictionary or a YAML file. How you define your sweep configuration depends on how you want to manage your sweep. \n\n:::info\nDefine your sweep configuration in a YAML file if you want to initialize a sweep and start a sweep agent from the command line. Define your sweep in a Python dictionary if you initialize a sweep and start a sweep entirely within a Python script or Jupyter notebook.\n::: \n\nThe following guide describes how to format your sweep configuration. See Sweep configuration options for a comprehensive list of top-level sweep configuration keys. \n\n## Basic structure \n\n## Double nested parameters \n\n## Sweep configuration template", "matching_document_document_id": "b44010f0-f785-4c17-a0c2-a3d6f73cea16", "query_weight": 0.7896774193548387 }, { "query_id": "e28dcfa9-b275-4509-94f6-379623fa5a58", "query": "Version control for large video datasets in machine learning projects", "matching_document": "## Version Control in Machine Learning\n### Data Version Control\n* Data preprocessing (data content changes) such as data cleaning, outlier handling, filling of missing values, etc.\n* Feature engineering (data becomes \"wider\") such as aggregation features, label encoding, scaling, etc.\n* Dataset splits (data is partitioned) typically mean dividing your data into training, validation, and testing data.\n* Dataset update (data becomes \"longer\") when new data points are available.", "matching_document_document_id": "152cb478-f767-4f0f-b64b-7a86691a728b", "query_weight": 1.904516129032258 }, { "query_id": "6b5e3890-b7d2-4c0d-b654-40d037dc782f", "query": "Impact of dataset updates on artifact versioning in ML experiments", "matching_document": "## Dataset Versioning\n### Iteratively refine your data\n\nAs you browse through your training data or add new batches of examples, you may notice issues like \n\n* incorrect ground truth labels\n* hard negatives or commonly misclassified examples\n* problematic class imbalances \n\nTo clean up and refine your data, you might modify incorrect labels, add or remove files to address imbalances, or group hard negatives into a special test split. With Artifacts, once you've finished a batch of changes, you can then call `run.log_artifact()` to push the new version to the cloud. This will automatically update the artifact with a new version to reflect your changes, while preserving the lineage and history of previous changes. \n\nYou can tag artifact versions with custom aliases, take notes on what changed, store metadata alongside each version, and see which experiment runs use a particular version. That way, your entire team can be sure they're working with the `latest` or a `stable` version of the data, to taste.", "matching_document_document_id": "9398af2a-2b45-43da-abf2-5cac450cee6b", "query_weight": 1.904516129032258 }, { "query_id": "5a31c448-b669-4555-97ac-0f75f064e7e1", "query": "dataset change impact on artifact versioning", "matching_document": "## Dataset Versioning\n### Iteratively refine your data\n\nAs you browse through your training data or add new batches of examples, you may notice issues like \n\n* incorrect ground truth labels\n* hard negatives or commonly misclassified examples\n* problematic class imbalances \n\nTo clean up and refine your data, you might modify incorrect labels, add or remove files to address imbalances, or group hard negatives into a special test split. With Artifacts, once you've finished a batch of changes, you can then call `run.log_artifact()` to push the new version to the cloud. This will automatically update the artifact with a new version to reflect your changes, while preserving the lineage and history of previous changes. \n\nYou can tag artifact versions with custom aliases, take notes on what changed, store metadata alongside each version, and see which experiment runs use a particular version. That way, your entire team can be sure they're working with the `latest` or a `stable` version of the data, to taste.", "matching_document_document_id": "9398af2a-2b45-43da-abf2-5cac450cee6b", "query_weight": 1.904516129032258 }, { "query_id": "032cdd50-a6db-4537-bc4a-a3a2ebb0d55b", "query": "Concatenating data tables from different artifact versions", "matching_document": "## Log Tables\n### Advanced: Artifact Tables\n#### Join Artifact Tables\n| Args | Description |\n| --- | --- |\n| table\\_1 | (str, `wandb.Table`, ArtifactEntry) the path to a `wandb.Table` in an artifact, the table object, or ArtifactEntry |\n| table\\_2 | (str, `wandb.Table`, ArtifactEntry) the path to a `wandb.Table` in an artifact, the table object, or ArtifactEntry |\n| join\\_key | (str, [str, str]) key or keys on which to perform the join | \n\nTo join two Tables you have logged previously in an artifact context, fetch them from the artifact and join the result into a new Table. \n\nFor example, demonstrates how to read one Table of original songs called `'original_songs'` and another Table of synthesized versions of the same songs called `'synth_songs'`. The proceeding code example joins the two tables on `\"song_id\"`, and uploads the resulting table as a new W&B Table:", "matching_document_document_id": "af7c008c-c910-4e5f-8952-d2d4d6c54bb7", "query_weight": 1.904516129032258 }, { "query_id": "0da7301e-03e0-49d0-abc9-baa0d4515523", "query": "WandB API access sweep best run example", "matching_document": "## Sweep\n\nView source on GitHub \n\nA set of runs associated with a sweep. \n\n```\nSweep(\nclient, entity, project, sweep\\_id, attrs=None\n)\n\n``` \n\n#### Examples: \n\nInstantiate with: \n\n```\napi = wandb.Api()\nsweep = api.sweep(path/to/sweep)\n\n``` \n\n| Attributes | |\n| --- | --- |\n| `runs` | (`Runs`) list of runs |\n| `id` | (str) sweep id |\n| `project` | (str) name of project |\n| `config` | (str) dictionary of sweep configuration |\n| `state` | (str) the state of the sweep |\n| `expected_run_count` | (int) number of expected runs for the sweep | \n\n## Methods \n\n### `best_run` \n\nView source \n\n```\nbest\\_run(\norder=None\n)\n\n``` \n\nReturn the best run sorted by the metric defined in config or the order passed in. \n\n### `display` \n\nView source \n\n```\ndisplay(\nheight=420, hidden=(False)\n) -> bool\n\n``` \n\nDisplay this object in jupyter. \n\n### `get` \n\nView source \n\n```\n@classmethod\nget(\nclient, entity=None, project=None, sid=None, order=None, query=None, \\*\\*kwargs\n)\n\n``` \n\n### `load` \n\nView source", "matching_document_document_id": "22d1890a-6b8a-47e3-93e5-47ad68eb9c99", "query_weight": 0.6503225806451612 }, { "query_id": "007ac1d9-fb0a-4e5b-bb69-d4e9c761e4b2", "query": "Examples of accessing sweep best run programmatically in WandB", "matching_document": "## Sweep\n\nView source on GitHub \n\nA set of runs associated with a sweep. \n\n```\nSweep(\nclient, entity, project, sweep\\_id, attrs=None\n)\n\n``` \n\n#### Examples: \n\nInstantiate with: \n\n```\napi = wandb.Api()\nsweep = api.sweep(path/to/sweep)\n\n``` \n\n| Attributes | |\n| --- | --- |\n| `runs` | (`Runs`) list of runs |\n| `id` | (str) sweep id |\n| `project` | (str) name of project |\n| `config` | (str) dictionary of sweep configuration |\n| `state` | (str) the state of the sweep |\n| `expected_run_count` | (int) number of expected runs for the sweep | \n\n## Methods \n\n### `best_run` \n\nView source \n\n```\nbest\\_run(\norder=None\n)\n\n``` \n\nReturn the best run sorted by the metric defined in config or the order passed in. \n\n### `display` \n\nView source \n\n```\ndisplay(\nheight=420, hidden=(False)\n) -> bool\n\n``` \n\nDisplay this object in jupyter. \n\n### `get` \n\nView source \n\n```\n@classmethod\nget(\nclient, entity=None, project=None, sid=None, order=None, query=None, \\*\\*kwargs\n)\n\n``` \n\n### `load` \n\nView source", "matching_document_document_id": "22d1890a-6b8a-47e3-93e5-47ad68eb9c99", "query_weight": 0.6503225806451612 }, { "query_id": "d2438c8f-d60f-445d-b2e5-eef583b73527", "query": "WandB API link best run", "matching_document": "```\n# !pip install -U wandb\n\n``` \n\n```\nimport wandb\n\n# Initialize a wandb run\nwandb.init(project='model\\_registry\\_demo')\n\n# Create a dummy model file (this could be replaced with your actual model file)\nmodel\\_path = 'dummy\\_model.txt'\nwith open(model\\_path, 'w') as file:\nfile.write('This is a dummy model file.')\n\n# Log the model to WandB\nwandb.log\\_model(name=f\"model-{wandb.run.id}\", path=model\\_path, aliases=[\"best\", \"dev\"])\n\n``` \n\n```\n# Link the best model to the W&B Model Registry (e.g. after the training is finished)\nartifact\\_name = f\"model-{wandb.run.id}:best\"\nbest\\_model\\_path = wandb.use\\_model(artifact\\_name)\n\n# Link the best model to the registry\nwandb.link\\_model(path=best\\_model\\_path,\nregistered\\_model\\_name=\"My\\_ML\\_Task\",\naliases=[\"staging\"])\n\n``` \n\n```\nwandb.finish()\n\n``` \n\n```\n\n```", "matching_document_document_id": "ccf293f2-9f62-4233-a79b-d5d9a43fdf45", "query_weight": 0.6503225806451612 }, { "query_id": "bbeb7697-8335-42dd-b49c-664973418cb8", "query": "WandB sweep API guide", "matching_document": "## Sweep\n\nView source on GitHub \n\nA set of runs associated with a sweep. \n\n```\nSweep(\nclient, entity, project, sweep\\_id, attrs=None\n)\n\n``` \n\n#### Examples: \n\nInstantiate with: \n\n```\napi = wandb.Api()\nsweep = api.sweep(path/to/sweep)\n\n``` \n\n| Attributes | |\n| --- | --- |\n| `runs` | (`Runs`) list of runs |\n| `id` | (str) sweep id |\n| `project` | (str) name of project |\n| `config` | (str) dictionary of sweep configuration |\n| `state` | (str) the state of the sweep |\n| `expected_run_count` | (int) number of expected runs for the sweep | \n\n## Methods \n\n### `best_run` \n\nView source \n\n```\nbest\\_run(\norder=None\n)\n\n``` \n\nReturn the best run sorted by the metric defined in config or the order passed in. \n\n### `display` \n\nView source \n\n```\ndisplay(\nheight=420, hidden=(False)\n) -> bool\n\n``` \n\nDisplay this object in jupyter. \n\n### `get` \n\nView source \n\n```\n@classmethod\nget(\nclient, entity=None, project=None, sid=None, order=None, query=None, \\*\\*kwargs\n)\n\n``` \n\n### `load` \n\nView source", "matching_document_document_id": "22d1890a-6b8a-47e3-93e5-47ad68eb9c99", "query_weight": 0.6503225806451612 }, { "query_id": "0cca5c5c-8326-41b6-9b58-b0f4cc9448b3", "query": "using run_id securely Weights & Biases", "matching_document": "## How Weights and Biases Can Help with Audits & Regulatory Guidelines\n### A simple framework to structure your future experiments\n* W&B stores a copy of all the source files at the current time the run was logged. Consider this to be a snapshot of your code for ease of reproducibility - this is similar to GitHub commits, but IMHO, more convenient. If you're asked 6 months down the line about model configs and source code, then all you need is to refer to the weights and biases run_id to have access to everything. \n\n* All model weights after every epoch get stored with Weights and Biases. IMO, this is one of the most wonderful features of W&B. The ability to store model weights alongside runs means, that not only do you have access to source code, but also model weights at any given time. This is perfect for audit and compliance too! \n\n* As can be seen above, every artifact is associated to each run. In this case, the run is called ``charmed-haze-1`, I could now just click on the run, to go back to have access to all metrics, source code files and artifacts.", "matching_document_document_id": "ba046421-adb8-4803-b09a-5e8a4aaddb5d", "query_weight": 1.0219354838709678 }, { "query_id": "f5d9abe5-0dc6-43ea-ae59-49039991b0b6", "query": "I'd like to attach some images to my experiment run. I generate the images on a different host machine, but using the outputs of the current training run. How can I get this host machine to log images to the run that is happening on a different machine? I was thinking about sharing the run_id, but the docs seem to warn against this and recommend using groups. Groups seem like overkill for just needing to post images to the same run. Any suggestions?", "matching_document": "## How This Works\nwandb.log({'examles': wandb.Image(image)}) and then I'm using the wandb.Api to download them after training. Finally, I put them all together into gifs using [pillow](http://PILlow.readthedocs.io). \n\n## Log the Images During Training \n\nYou may know that you can log images ([and lots more!](https://docs.wandb.ai/guides/track/log/media)) to W&B during training. When you do that, you can see the latest images logged in your workspace and get a slider to see earlier images. \n\nYou can see an example of the code and the produced slider below: \n\n```\nimages = [PIL.Image.fromarray(image) for image in image_array]\nwandb.log({\"examples\": [wandb.Image(image) for image in images]}\n``` \n\n## Download the Images With the W&B API \n\nThe W&B API is a Python wrapper you can use to get data from W&B. \n\n```\nimport wandb\napi = wandb.Api()\nrun = api.run(\"//\")\n``` \n\n\ud83d\udc49 [Here's a guide to learn more about what you can do with the API.](https://docs.wandb.ai/ref/python/public-api)", "matching_document_document_id": "c06fe7f9-16b6-495d-9bf9-1ab47779c9eb", "query_weight": 1.0219354838709678 }, { "query_id": "f8dc983f-77d6-4b2b-ac64-cbe897d6f08f", "query": "is there an example of how to use Launch on Sagemaker using BYOI (Bring your own image)?", "matching_document": "## Set up for SageMaker\n### Configure a launch agent\nConfigure the launch agent with a YAML config file named `launch-config.yaml`. By default, W&B will check for the config file in `~/.config/wandb/launch-config.yaml`. You can optionally specify a different directory when you activate the launch agent. \n\nThe following YAML snippet demonstrates how to specify the core config agent options: \n\nlaunch-config.yaml\n```\nmax\\_jobs: \nqueues:\n- \n\n``` \n\n:::tip\nThere are two ways to submit launch on Amazon SageMaker:\n\\* Option 1: Bring your own image (BYOI) and push it to your Amazon ECR repository.\n\\* Option 2: Let the W&B Launch agent build a container for your and push it to your ECR repository. \n\nYou will need to provide additional information to your launch agent configuration if you want the launch agent to build images for you (Option 2). For more information, see Advanced agent set up.\n:::", "matching_document_document_id": "ba247dd5-abd4-4f66-98b3-8071cd2375a7", "query_weight": 0.6503225806451612 }, { "query_id": "aee7dd88-4bad-498b-82ab-0c3cf4257846", "query": "BYOI", "matching_document": "## Set up for SageMaker\n### Configure a launch agent\nConfigure the launch agent with a YAML config file named `launch-config.yaml`. By default, W&B will check for the config file in `~/.config/wandb/launch-config.yaml`. You can optionally specify a different directory when you activate the launch agent. \n\nThe following YAML snippet demonstrates how to specify the core config agent options: \n\nlaunch-config.yaml\n```\nmax\\_jobs: \nqueues:\n- \n\n``` \n\n:::tip\nThere are two ways to submit launch on Amazon SageMaker:\n\\* Option 1: Bring your own image (BYOI) and push it to your Amazon ECR repository.\n\\* Option 2: Let the W&B Launch agent build a container for your and push it to your ECR repository. \n\nYou will need to provide additional information to your launch agent configuration if you want the launch agent to build images for you (Option 2). For more information, see Advanced agent set up.\n:::", "matching_document_document_id": "ba247dd5-abd4-4f66-98b3-8071cd2375a7", "query_weight": 0.6503225806451612 }, { "query_id": "54acdca9-6c6e-4450-b814-bb039b459d95", "query": "How to configure AWS SageMaker for BYOI with Weights & Biases Launch?", "matching_document": "## Set up for SageMaker\n### Configure a launch agent\nConfigure the launch agent with a YAML config file named `launch-config.yaml`. By default, W&B will check for the config file in `~/.config/wandb/launch-config.yaml`. You can optionally specify a different directory when you activate the launch agent. \n\nThe following YAML snippet demonstrates how to specify the core config agent options: \n\nlaunch-config.yaml\n```\nmax\\_jobs: \nqueues:\n- \n\n``` \n\n:::tip\nThere are two ways to submit launch on Amazon SageMaker:\n\\* Option 1: Bring your own image (BYOI) and push it to your Amazon ECR repository.\n\\* Option 2: Let the W&B Launch agent build a container for your and push it to your ECR repository. \n\nYou will need to provide additional information to your launch agent configuration if you want the launch agent to build images for you (Option 2). For more information, see Advanced agent set up.\n:::", "matching_document_document_id": "ba247dd5-abd4-4f66-98b3-8071cd2375a7", "query_weight": 0.6503225806451612 }, { "query_id": "940827ef-dc60-42b7-be8f-42b5965d3532", "query": "Example configuration for integrating a custom image with Weights & Biases Launch on SageMaker.", "matching_document": "## Set up for SageMaker\n### Configure a launch agent\nConfigure the launch agent with a YAML config file named `launch-config.yaml`. By default, W&B will check for the config file in `~/.config/wandb/launch-config.yaml`. You can optionally specify a different directory when you activate the launch agent. \n\nThe following YAML snippet demonstrates how to specify the core config agent options: \n\nlaunch-config.yaml\n```\nmax\\_jobs: \nqueues:\n- \n\n``` \n\n:::tip\nThere are two ways to submit launch on Amazon SageMaker:\n\\* Option 1: Bring your own image (BYOI) and push it to your Amazon ECR repository.\n\\* Option 2: Let the W&B Launch agent build a container for your and push it to your ECR repository. \n\nYou will need to provide additional information to your launch agent configuration if you want the launch agent to build images for you (Option 2). For more information, see Advanced agent set up.\n:::", "matching_document_document_id": "ba247dd5-abd4-4f66-98b3-8071cd2375a7", "query_weight": 0.6503225806451612 }, { "query_id": "564efdbd-2a64-4465-9def-13e78fd60018", "query": "do you support yolov5? how can i log COCO bounding boxes?", "matching_document": "\" \n\n# Track and Debug Your YOLOv5 Models With Weights & Biases \n\nDescription: In this article, we introduce automatic bounding box debugging, system metrics, model performance metrics, and shareable, reproducible model training for YOLOv5, all using Weights & Biases. \n\nBody: \n\n[YOLOv5](https://wandb.ai/fully-connected/blog/yolov5) now comes with a native [Weights & Biases](http://wandb.com/) integration that tracks your model pipelines \u2013 including model performance, hyperparameters, GPU usage, predictions, and datasets. \n\n## Table of Contents \n\n# Getting Started \n\nYour model metrics are automatically tracked by YOLOv5 if you have wandb installed. \n\n* To install W&B, run `pip install wandb`.\n* After `wandb` is installed, you can run any YOLOv5 training script as you normally would. For example: \n\n```\npython train.py --batch 16 --epochs 500 --data coco128.yaml --cfg yolov5s.yaml\n``` \n\n# Utilities you get out-of-the-box with W&B \n\n## 1. The Dashboard \n\n## 2. Bounding Box Debugging", "matching_document_document_id": "cc4d8db7-4c7b-46ea-b03c-32ed58917f52", "query_weight": 1.8116129032258064 }, { "query_id": "d8168c5f-2c69-4ff4-9b30-1a9fc07a9888", "query": "How to use Weights & Biases groups for logging images?", "matching_document": "## Weights & Biases Grouping\n\nFrom your script, use grouping to organize individual runs into larger experiments. This is useful for distributed training and cross validation. \n\nIn `wandb.init()`:\n- **group**: the first level of organization, usually this is your unique experiment name\n- **job\\_type**: the second level of grouping, this is often `train`, `eval`, `optimizer`, `rollout` etc. \n\n**Links**\n- Documentation\n- Example project\n- Example dedicated group page \n\n```\n!pip install wandb -qqq\n\n``` \n\nIn this example, I'm setting the experiment index up front, and then incrementing it every time I re-run the cell below. \n\n```\n# Set experiment index (for demo purposes)\nexperiment_index = 1\n\n```", "matching_document_document_id": "9a74425c-7291-452b-969d-1fe385cadf77", "query_weight": 1.8116129032258064 }, { "query_id": "b80ca014-84e7-4177-95a0-462338d420ae", "query": "Integrating Weights & Biases with training scripts for metadata logging", "matching_document": "## Add W&B to your code\n#### Original training script\n```python showLineNumbers\n\n# train.py\n\nimport random\nimport numpy as np\n\ndef train\\_one\\_epoch(epoch, lr, bs):\nacc = 0.25 + ((epoch / 30) + (random.random() / 10))\nloss = 0.2 + (1 - ((epoch - 1) / 10 + random.random() / 5))\nreturn acc, loss\n\ndef evaluate\\_one\\_epoch(epoch):\nacc = 0.1 + ((epoch / 20) + (random.random() / 10))\nloss = 0.25 + (1 - ((epoch - 1) / 10 + random.random() / 6))\nreturn acc, loss\n\nconfig = {\"lr\": 0.0001, \"bs\": 16, \"epochs\": 5}\n\ndef main():\n# Note that we define values from `wandb.config`\n# instead of defining hard values\nlr = config[\"lr\"]\nbs = config[\"bs\"]\nepochs = config[\"epochs\"]\n\n```\nfor epoch in np.arange(1, epochs):\ntrain_acc, train_loss = train_one_epoch(epoch, lr, bs)\nval_acc, val_loss = evaluate_one_epoch(epoch) \n\nprint(\"epoch: \", epoch)\nprint(\"training accuracy:\", train_acc, \"training loss:\", train_loss)\nprint(\"validation accuracy:\", val_acc, \"training loss:\", val_loss) \n\n```\n\n# Call the main function.\n\nmain()\n\n```", "matching_document_document_id": "5ae0de15-7b29-4388-b69e-895cd8da85bc", "query_weight": 1.8116129032258064 }, { "query_id": "e9a0cc1d-11f6-4912-a406-b35f7b98ee77", "query": "What is the purpose of using 'datum' in custom charts?", "matching_document": "## Customized Usage\nHere are some simple ones: \n\n* change the appearance and opacity of the point markers: change `\"mark\": {\"type\": \"circle\",` to `\"mark\": {\"type\": \"point\", \"opacity\" : 0.3,`\n* rename the axis titles for clarity: add `\"title\" : \"Your Title\"` to the `x` and `y` fields under `encoding` \n\nHere is a pretty advanced one: set opacity conditionally based on the prefix of your run names, so you can display two different run sets in one-panel section (previously impossible in W&B). You will need to create two copies of the panel and the Vega spec and modify the test condition in one of the copies to make the inverse set of runs invisible (i.e. replace \"frog\" with \"bug\"). \n\n```\n\"opacity\" : {\n\"condition\" :{\n\"test\" : \"indexof(datum.name, 'frog') > -1\",\n\"value\" : 0}\n}\n}\n```", "matching_document_document_id": "285ab3e2-c463-43d7-ab93-3f1d7fa0037c", "query_weight": 0.8361290322580645 }, { "query_id": "19eef18e-4c29-4293-940f-63c774660809", "query": "datum function in W&B charts", "matching_document": "## Customized Usage\nHere are some simple ones: \n\n* change the appearance and opacity of the point markers: change `\"mark\": {\"type\": \"circle\",` to `\"mark\": {\"type\": \"point\", \"opacity\" : 0.3,`\n* rename the axis titles for clarity: add `\"title\" : \"Your Title\"` to the `x` and `y` fields under `encoding` \n\nHere is a pretty advanced one: set opacity conditionally based on the prefix of your run names, so you can display two different run sets in one-panel section (previously impossible in W&B). You will need to create two copies of the panel and the Vega spec and modify the test condition in one of the copies to make the inverse set of runs invisible (i.e. replace \"frog\" with \"bug\"). \n\n```\n\"opacity\" : {\n\"condition\" :{\n\"test\" : \"indexof(datum.name, 'frog') > -1\",\n\"value\" : 0}\n}\n}\n```", "matching_document_document_id": "285ab3e2-c463-43d7-ab93-3f1d7fa0037c", "query_weight": 0.8361290322580645 }, { "query_id": "04262775-8f9d-463e-af75-2c6f4dd599d7", "query": "step by step guide on datum in W&B custom charts", "matching_document": "## Customized Usage\nHere are some simple ones: \n\n* change the appearance and opacity of the point markers: change `\"mark\": {\"type\": \"circle\",` to `\"mark\": {\"type\": \"point\", \"opacity\" : 0.3,`\n* rename the axis titles for clarity: add `\"title\" : \"Your Title\"` to the `x` and `y` fields under `encoding` \n\nHere is a pretty advanced one: set opacity conditionally based on the prefix of your run names, so you can display two different run sets in one-panel section (previously impossible in W&B). You will need to create two copies of the panel and the Vega spec and modify the test condition in one of the copies to make the inverse set of runs invisible (i.e. replace \"frog\" with \"bug\"). \n\n```\n\"opacity\" : {\n\"condition\" :{\n\"test\" : \"indexof(datum.name, 'frog') > -1\",\n\"value\" : 0}\n}\n}\n```", "matching_document_document_id": "285ab3e2-c463-43d7-ab93-3f1d7fa0037c", "query_weight": 0.8361290322580645 }, { "query_id": "fdd9f0a9-3581-4ab9-a75d-329c2d254390", "query": "using datum for data visualization in W&B", "matching_document": "## Customized Usage\nHere are some simple ones: \n\n* change the appearance and opacity of the point markers: change `\"mark\": {\"type\": \"circle\",` to `\"mark\": {\"type\": \"point\", \"opacity\" : 0.3,`\n* rename the axis titles for clarity: add `\"title\" : \"Your Title\"` to the `x` and `y` fields under `encoding` \n\nHere is a pretty advanced one: set opacity conditionally based on the prefix of your run names, so you can display two different run sets in one-panel section (previously impossible in W&B). You will need to create two copies of the panel and the Vega spec and modify the test condition in one of the copies to make the inverse set of runs invisible (i.e. replace \"frog\" with \"bug\"). \n\n```\n\"opacity\" : {\n\"condition\" :{\n\"test\" : \"indexof(datum.name, 'frog') > -1\",\n\"value\" : 0}\n}\n}\n```", "matching_document_document_id": "285ab3e2-c463-43d7-ab93-3f1d7fa0037c", "query_weight": 0.8361290322580645 }, { "query_id": "cc8f6990-3d26-4bbb-b2f3-190efef0adf2", "query": "implementing datum in visualization", "matching_document": "## Customized Usage\nHere are some simple ones: \n\n* change the appearance and opacity of the point markers: change `\"mark\": {\"type\": \"circle\",` to `\"mark\": {\"type\": \"point\", \"opacity\" : 0.3,`\n* rename the axis titles for clarity: add `\"title\" : \"Your Title\"` to the `x` and `y` fields under `encoding` \n\nHere is a pretty advanced one: set opacity conditionally based on the prefix of your run names, so you can display two different run sets in one-panel section (previously impossible in W&B). You will need to create two copies of the panel and the Vega spec and modify the test condition in one of the copies to make the inverse set of runs invisible (i.e. replace \"frog\" with \"bug\"). \n\n```\n\"opacity\" : {\n\"condition\" :{\n\"test\" : \"indexof(datum.name, 'frog') > -1\",\n\"value\" : 0}\n}\n}\n```", "matching_document_document_id": "285ab3e2-c463-43d7-ab93-3f1d7fa0037c", "query_weight": 0.8361290322580645 }, { "query_id": "559fb01b-ff35-4149-802b-bda87f02ed70", "query": "how do I fix an error with wandb Table construction from pandas dataframe: TypeError: Data row contained incompatible types", "matching_document": "## Table\n```\nimport numpy as np\nimport wandb\n\nwandb.init()\n\ntbl = wandb.Table(columns=[\"image\", \"label\"])\n\nimages = np.random.randint(0, 255, [2, 100, 100, 3], dtype=np.uint8)\nlabels = [\"panda\", \"gibbon\"]\n[tbl.add\\_data(wandb.Image(image), label) for image, label in zip(images, labels)]\n\nwandb.log({\"classifier\\_out\": tbl})\n\n``` \n\nTables added directly to runs as above will produce a corresponding Table Visualizer in the\nWorkspace which can be used for further analysis and exporting to reports. \n\nTables added to artifacts can be viewed in the Artifact Tab and will render\nan equivalent Table Visualizer directly in the artifact browser. \n\nTables expect each value for a column to be of the same type. By default, a column supports\noptional values, but not mixed values. If you absolutely need to mix types,\nyou can enable the `allow_mixed_types` flag which will disable type checking\non the data. This will result in some table analytics features being disabled\ndue to lack of consistent typing.", "matching_document_document_id": "33a1b667-9707-4e12-b07f-0bfba6136ff7", "query_weight": 0.8361290322580645 }, { "query_id": "38d125ae-a1d2-4594-865d-a347b68b0b6b", "query": "Best practices for converting pandas DataFrame to wandb Table", "matching_document": "## Plotting with Pandas and Weights & Biases: Step-by-Step Guide\n### Integration of Weights & Biases With Pandas for Interactive and Collaborative Data Plotting\n\nIt is also extremely simple to log any Pandas dataframe to a workspace by converting it into a W&B Table: \n\n```\nimport wandb\nimport pandas as pd\n\n# Read our CSV into a new DataFrame\npandas_dataframe = pd.read_csv(\"data.csv\")\n\n# Convert the DataFrame into a W&B Table\nwandb_table = wandb.Table(dataframe=pandas_dataframe)\n\n# Add the table to an Artifact to increase the row\nwandb_table_artifact = wandb.Artifact(\n\"wandb_artifact\",\ntype=\"dataset\")\nwandb_table_artifact.add(wandb_table, \"table\")\n\n# Log the raw csv file within an artifact to preserve our data\nwandb_table_artifact.add_file(\"data.csv\")\n\n# Start a W&B run to log data\nrun = wandb.init(project=\"...\")\n\n# Log the table to visualize with a run...\nrun.log({\"data\": wandb_table})\n\n# and Log as an Artifact\nrun.log_artifact(wandb_table_artifact)\n```", "matching_document_document_id": "b584a5a0-834b-4d77-b8e3-86ce64c829d1", "query_weight": 0.8361290322580645 }, { "query_id": "c3296450-946c-41e8-b266-872c46f3582d", "query": "How to ensure pandas DataFrame types are compatible with wandb Table?", "matching_document": "\"\"\"The Table class used to display and analyze tabular data.\n\n Unlike traditional spreadsheets, Tables support numerous types of data:\n scalar values, strings, numpy arrays, and most subclasses of `wandb.data_types.Media`.\n This means you can embed `Images`, `Video`, `Audio`, and other sorts of rich, annotated media\n directly in Tables, alongside other traditional scalar values.\n\n This class is the primary class used to generate the Table Visualizer\n in the UI: https://docs.wandb.ai/guides/data-vis/tables.\n\n Arguments:\n columns: (List[str]) Names of the columns in the table.\n Defaults to [\"Input\", \"Output\", \"Expected\"].\n data: (List[List[any]]) 2D row-oriented array of values.\n dataframe: (pandas.DataFrame) DataFrame object used to create the table.\n When set, `data` and `columns` arguments are ignored.\n optional: (Union[bool,List[bool]]) Determines if `None` values are allowed. Default to True\n - If a singular bool value, then the optionality is enforced for all\n columns specified at construction time\n - If a list of bool values, then the optionality is applied to each\n column - should be the same length as `columns`\n applies to all columns. A list of bool values applies to each respective column.\n allow_mixed_types: (bool) Determines if columns are allowed to have mixed types\n (disables type validation). Defaults to False\n \"\"\"", "matching_document_document_id": "d7e97d5b-c188-4b37-80a8-2c514ca8081c", "query_weight": 0.8361290322580645 }, { "query_id": "68317306-33cf-458e-86dc-127f030f9987", "query": "Common errors when creating wandb Tables from pandas DataFrames and how to fix them", "matching_document": "## Table\n```\nimport numpy as np\nimport wandb\n\nwandb.init()\n\ntbl = wandb.Table(columns=[\"image\", \"label\"])\n\nimages = np.random.randint(0, 255, [2, 100, 100, 3], dtype=np.uint8)\nlabels = [\"panda\", \"gibbon\"]\n[tbl.add\\_data(wandb.Image(image), label) for image, label in zip(images, labels)]\n\nwandb.log({\"classifier\\_out\": tbl})\n\n``` \n\nTables added directly to runs as above will produce a corresponding Table Visualizer in the\nWorkspace which can be used for further analysis and exporting to reports. \n\nTables added to artifacts can be viewed in the Artifact Tab and will render\nan equivalent Table Visualizer directly in the artifact browser. \n\nTables expect each value for a column to be of the same type. By default, a column supports\noptional values, but not mixed values. If you absolutely need to mix types,\nyou can enable the `allow_mixed_types` flag which will disable type checking\non the data. This will result in some table analytics features being disabled\ndue to lack of consistent typing.", "matching_document_document_id": "33a1b667-9707-4e12-b07f-0bfba6136ff7", "query_weight": 0.8361290322580645 }, { "query_id": "983c454b-1fcc-4ce8-a28e-a6d6408cd43d", "query": "efficient logging practices", "matching_document": "## \ud83d\udccf Best Practices\n\n1. **Projects**: Log multiple runs to a project to compare them. `wandb.init(project=\"project-name\")`\n2. **Groups**: For multiple processes or cross validation folds, log each process as a runs and group them together. `wandb.init(group='experiment-1')`\n3. **Tags**: Add tags to track your current baseline or production model.\n4. **Notes**: Type notes in the table to track the changes between runs.\n5. **Reports**: Take quick notes on progress to share with colleagues and make dashboards and snapshots of your ML projects.", "matching_document_document_id": "f9c5a05a-9075-4bb6-9a41-1ca20631ddda", "query_weight": 0.5109677419354839 }, { "query_id": "a1963394-454e-4627-a101-0c1e8bc82d8b", "query": "training process metric logging", "matching_document": "## Train\n\n## Logging training and validation metrics \n\nWe are ready to train our model. We will log our training and validation loss/metrics, and learning rate to W&B with a simple wandb.log() method. wandb.log() method is very powerful and can log things ranging from scalar values, histograms, plots, images, and tables to 3D objects. You can refer to the [documentation](https://docs.wandb.ai/guides/track/log) to learn all the things you can log with wandb.log(). \n\nWe'll also save the model with the best validation dice metric score and log the best metric value and the epoch value at which it was logged to W&B. You'll see that code below. \n\n> \n\n## Logging the model gradients \n\nYou might have noticed the following line in the above code block: \n\n```\n# \ud83d\udc1d log gradients of the model to wandb\nwandb.watch(model, log_freq=100)\n``` \n\n## System Metrics \n\nYou can also view the system metrics like CPU/GPU utilization, GPU Temperature, network traffic, etc. \n\n---", "matching_document_document_id": "278eea52-5365-464c-b0b0-72d2d1be3043", "query_weight": 0.5109677419354839 }, { "query_id": "06c62297-651c-481c-8f8b-30483a3f756c", "query": "artifact logging best practices", "matching_document": "## 1\ufe0f\u20e3 Log a Dataset\nThis sets up a pattern we'll see repeated in this example:\nthe code to log the data as an Artifact is wrapped around the code for\nproducing that data.\nIn this case, the code for `load`ing the data is\nseparated out from the code for `load_and_log`ging the data. \n\nThis is good practice! \n\nIn order to log these datasets as Artifacts,\nwe just need to\n1. create a `Run` with `wandb.init`, (L4)\n2. create an `Artifact` for the dataset (L10), and\n3. save and log the associated `file`s (L20, L23). \n\nCheck out the example the code cell below\nand then expand the sections afterwards for more details.", "matching_document_document_id": "18c31b0c-33a8-4cea-9c6c-6c47f7a8d176", "query_weight": 0.5109677419354839 }, { "query_id": "95aba451-82de-408b-90ff-1303472156c1", "query": "account data privacy", "matching_document": "## Data\n\n## AWS S3, Google Cloud Storage and W&B Artifacts \n\nWeights and Biases **Artifacts** enable you to log end-to-end training pipelines to ensure your experiments are always reproducible. \n\nData privacy is critical to Weights & Biases and so we support the creation of Artifacts from reference locations such as your own private cloud such as AWS S3 or Google Cloud Storage. Local, on-premises of W&B are also available upon request. \n\nBy default, W&B stores artifact files in a private Google Cloud Storage bucket located in the United States. All files are encrypted at rest and in transit. For sensitive files, we recommend a private W&B installation or the use of reference artifacts. \n\n##\u00a0Artifacts Reference Example\n**Create an artifact with the S3/GCS metadata** \n\nThe artifact only consists of metadata about the S3/GCS object such as its ETag, size, and version ID (if object versioning is enabled on the bucket). \n\n## Login to W&B \n\nLogin to Weights and Biases \n\n## Vehicle Loan Dataset", "matching_document_document_id": "15a6bbe9-03fd-4f70-89f9-3a0ad32cc614", "query_weight": 0.9754838709677419 }, { "query_id": "b38a75b5-1575-4538-8209-9ea02656172c", "query": "Is there a way to delete all experiments and projects associated with my account?", "matching_document": "## Project Page\n### Frequently Asked Questions\n\n### How can I delete projects? \n\nYou can delete your project by clicking the three dots on the right of the overview tab. \n\nIf the project is empty (i.e. it has no runs), you can delete it by clicking the dropdown menu in the top-right and selecting \"Delete project\". \n\n### Where are the privacy settings for projects? How can I make a project public or private? \n\nClick the lock in the navigation bar at the top of the page to change project privacy settings. You can edit who can view or submit runs to your project. These settings include all runs and reports in the project. If you'd like to share your results with just a few people, you can create a private team. \n\n### How do I reset my workspace? \n\nIf you see an error like the one below on your project page, here's how to reset your workspace.`\"objconv: \"100000000000\" overflows the maximum values of a signed 64 bits integer\"`", "matching_document_document_id": "6c7a754d-2ba6-4b58-a1a7-29aecc2eae09", "query_weight": 0.9754838709677419 }, { "query_id": "c3bb69b9-120c-461e-abeb-6ab4b8bf536d", "query": "exporting CSV files web panel", "matching_document": "\" \n\n# Export Your Data from W&B \n\nDescription: Learn how to export CSV, PDF, and LaTeX from W&B \n\nBody: \n\nWe're excited to announce our latest highly requested feature! You can now export tables, graphs and reports to CSV, LaTeX and PDF. Here are some examples of how you can export data easily from the Weights & Biases UI. \n\nIn any graph, click on the arrow in the upper right and click \u201cPanel Export\u201d. \n\nYou\u2019ll see a popup table with the underlying data - click Save as CSV to start the CSV download. \n\nIn a table click the \u201cExport Table\u201d button in the upper right corner. \n\nIn a report, click the \u201cDownload as LaTeX\u201d link. \n\nThis will download a zip file with a report.tex and images of your panels attached. The .tex file is a latex file that you can modify to style your document. You can load the zip file into Overleaf.com and use their visual editor, or you can download pdflatex and run \u2018pdflatext report.tex\u2019 to generate a report.pdf file. \n\nHere\u2019s an example of an exported LaTeX report: \n\n\"", "matching_document_document_id": "78c84355-d5cd-4b5f-920d-509ab7bf3ad9", "query_weight": 0.8361290322580645 }, { "query_id": "72806434-2cfc-48b5-a070-97293fd2a493", "query": "web panel data export", "matching_document": "\" \n\n# Export Your Data from W&B \n\nDescription: Learn how to export CSV, PDF, and LaTeX from W&B \n\nBody: \n\nWe're excited to announce our latest highly requested feature! You can now export tables, graphs and reports to CSV, LaTeX and PDF. Here are some examples of how you can export data easily from the Weights & Biases UI. \n\nIn any graph, click on the arrow in the upper right and click \u201cPanel Export\u201d. \n\nYou\u2019ll see a popup table with the underlying data - click Save as CSV to start the CSV download. \n\nIn a table click the \u201cExport Table\u201d button in the upper right corner. \n\nIn a report, click the \u201cDownload as LaTeX\u201d link. \n\nThis will download a zip file with a report.tex and images of your panels attached. The .tex file is a latex file that you can modify to style your document. You can load the zip file into Overleaf.com and use their visual editor, or you can download pdflatex and run \u2018pdflatext report.tex\u2019 to generate a report.pdf file. \n\nHere\u2019s an example of an exported LaTeX report: \n\n\"", "matching_document_document_id": "78c84355-d5cd-4b5f-920d-509ab7bf3ad9", "query_weight": 0.8361290322580645 }, { "query_id": "19c3b949-ba1b-4bbc-9e1f-5a10f9a2f78c", "query": "exporting CSV files", "matching_document": "\" \n\n# Export Your Data from W&B \n\nDescription: Learn how to export CSV, PDF, and LaTeX from W&B \n\nBody: \n\nWe're excited to announce our latest highly requested feature! You can now export tables, graphs and reports to CSV, LaTeX and PDF. Here are some examples of how you can export data easily from the Weights & Biases UI. \n\nIn any graph, click on the arrow in the upper right and click \u201cPanel Export\u201d. \n\nYou\u2019ll see a popup table with the underlying data - click Save as CSV to start the CSV download. \n\nIn a table click the \u201cExport Table\u201d button in the upper right corner. \n\nIn a report, click the \u201cDownload as LaTeX\u201d link. \n\nThis will download a zip file with a report.tex and images of your panels attached. The .tex file is a latex file that you can modify to style your document. You can load the zip file into Overleaf.com and use their visual editor, or you can download pdflatex and run \u2018pdflatext report.tex\u2019 to generate a report.pdf file. \n\nHere\u2019s an example of an exported LaTeX report: \n\n\"", "matching_document_document_id": "78c84355-d5cd-4b5f-920d-509ab7bf3ad9", "query_weight": 0.8361290322580645 }, { "query_id": "d42470cc-38c1-45c7-97d2-15d4e1d01964", "query": "Example scripts with wandb.finish() correctly implemented", "matching_document": "```\n# !pip install -U wandb\n\n``` \n\n```\nimport wandb\n\n# Initialize a wandb run\nwandb.init(project='model\\_registry\\_demo')\n\n# Create a dummy model file (this could be replaced with your actual model file)\nmodel\\_path = 'dummy\\_model.txt'\nwith open(model\\_path, 'w') as file:\nfile.write('This is a dummy model file.')\n\n# Log the model to WandB\nwandb.log\\_model(name=f\"model-{wandb.run.id}\", path=model\\_path, aliases=[\"best\", \"dev\"])\n\n``` \n\n```\n# Link the best model to the W&B Model Registry (e.g. after the training is finished)\nartifact\\_name = f\"model-{wandb.run.id}:best\"\nbest\\_model\\_path = wandb.use\\_model(artifact\\_name)\n\n# Link the best model to the registry\nwandb.link\\_model(path=best\\_model\\_path,\nregistered\\_model\\_name=\"My\\_ML\\_Task\",\naliases=[\"staging\"])\n\n``` \n\n```\nwandb.finish()\n\n``` \n\n```\n\n```", "matching_document_document_id": "ccf293f2-9f62-4233-a79b-d5d9a43fdf45", "query_weight": 0.6967741935483871 }, { "query_id": "87e6fc94-9bd4-4cbe-bb4d-0214bd776024", "query": "wandb.finish() placement", "matching_document": "```\n# !pip install -U wandb\n\n``` \n\n```\nimport wandb\n\n# Initialize a wandb run\nwandb.init(project='model\\_registry\\_demo')\n\n# Create a dummy model file (this could be replaced with your actual model file)\nmodel\\_path = 'dummy\\_model.txt'\nwith open(model\\_path, 'w') as file:\nfile.write('This is a dummy model file.')\n\n# Log the model to WandB\nwandb.log\\_model(name=f\"model-{wandb.run.id}\", path=model\\_path, aliases=[\"best\", \"dev\"])\n\n``` \n\n```\n# Link the best model to the W&B Model Registry (e.g. after the training is finished)\nartifact\\_name = f\"model-{wandb.run.id}:best\"\nbest\\_model\\_path = wandb.use\\_model(artifact\\_name)\n\n# Link the best model to the registry\nwandb.link\\_model(path=best\\_model\\_path,\nregistered\\_model\\_name=\"My\\_ML\\_Task\",\naliases=[\"staging\"])\n\n``` \n\n```\nwandb.finish()\n\n``` \n\n```\n\n```", "matching_document_document_id": "ccf293f2-9f62-4233-a79b-d5d9a43fdf45", "query_weight": 0.6967741935483871 }, { "query_id": "80143a31-be96-4fec-8ea6-2ab482b2f8e1", "query": "wandb.finish() usage", "matching_document": "## \ud83c\udf31 Advanced Usage\n```\n# Initialize the W&B run\nrun = wandb.init(project='medmnist-bloodmnist', config=configs, job_type='train')\nconfig = wandb.config\n\n# Get validation table\ndata_art = run.use_artifact('ayush-thakur/medmnist-bloodmnist/medmnist_bloodmnist_dataset:latest', type='dataset')\nvalid_table = data_art.get(\"valid_data\")\n\n# Create a class table\nclass_table = wandb.Table(columns=[], data=[])\nclass_table.add_column(\"class_name\", list(config.class_names.values()))\n\n# Define WandbCallback for experiment tracking\nwandb_callback = WandbCallback(\nlog_evaluation=True,\nvalidation_row_processor=lambda ndx, row: validation_processor(ndx, row),\nprediction_row_processor=lambda ndx, row: prediction_processor(ndx, row),\nvalidation_steps=4,\nsave_model=False\n)\n\n# callbacks\ncallbacks = [earlystopper, wandb_callback, lr_callback]\n\n# Train\nmodel = train(config, callbacks=callbacks, verbose=1)\n\n# Evaluate the trained model\nloss, acc = model.evaluate(validloader)\nwandb.log({'evaluate/accuracy': acc})\n\n# Close the W&B run.\nwandb.finish()\n\n```", "matching_document_document_id": "c960fe5b-ee0d-4088-8d03-cf9beb3f5a79", "query_weight": 0.6967741935483871 }, { "query_id": "2639daca-f18b-4ec0-99d7-fbb9819b259a", "query": "Best practices for integrating wandb.finish() in Python code", "matching_document": "## Hugging Face Transformers\n### Getting started: track experiments\n#### 6) Finish your W&B Run (Notebook only)\n\nIf your training is encapsulated in a Python script, the W&B run will end when your script finishes. \n\nIf you are using a Jupyter or Google Colab notebook, you'll need to tell us when you're done with training by calling `wandb.finish()`. \n\n```\ntrainer.train() # start training and logging to W&B\n\n# post-training analysis, testing, other logged code\n\nwandb.finish()\n\n```", "matching_document_document_id": "6edbb7d2-323e-49de-9ca2-f289602fbef0", "query_weight": 0.6967741935483871 }, { "query_id": "144ecb9e-9eb2-4ed6-86d7-100908d77e5c", "query": "Removing wandb hooks from PyTorch models after run completion", "matching_document": "def unwatch(models=None):\n \"\"\"Remove pytorch model topology, gradient and parameter hooks.\n\n Args:\n models: (list) Optional list of pytorch models that have had watch called on them\n \"\"\"\n if models:\n if not isinstance(models, (tuple, list)):\n models = (models,)\n for model in models:\n if not hasattr(model, \"_wandb_hook_names\"):\n wandb.termwarn(\"%s model has not been watched\" % model)\n else:\n for name in model._wandb_hook_names:\n wandb.run._torch.unhook(name)\n delattr(model, \"_wandb_hook_names\")\n # TODO: we should also remove recursively model._wandb_watch_called\n\n else:\n wandb.run._torch.unhook_all()", "matching_document_document_id": "82e6e3a9-d9f0-4a47-b3ef-66fbe33dc419", "query_weight": 0.6038709677419355 }, { "query_id": "d9883a0a-c167-460c-84f4-944fd7b51d41", "query": "PyTorch model clean up after wandb", "matching_document": "## \u26a1 Pytorch Lightning models with Weights & Biases\n### \ud83c\udfcb\ufe0f\u200d Train Your Model\n\n```\ntrainer = Trainer(\nlogger=wandb_logger, # W&B integration\ncallbacks=[log_predictions_callback, # logging of sample predictions\ncheckpoint_callback], # our model checkpoint callback\naccelerator=\"gpu\", # use GPU\nmax_epochs=5) # number of epochs\n\n``` \n\n```\ntrainer.fit(model, training_loader, validation_loader)\n\n``` \n\nWhen we want to close our W&B run, we call `wandb.finish()` (mainly useful in notebooks, called automatically in scripts). \n\n```\nwandb.finish()\n\n``` \n\nWe can monitor losses, metrics, gradients, parameters and sample predictions as the model trains.", "matching_document_document_id": "49ac1158-602d-4a5b-bebb-8db75248b9e6", "query_weight": 0.6038709677419355 }, { "query_id": "612fe4df-1508-4b43-b009-2c1fb0c133aa", "query": "detach wandb hooks from torch modules", "matching_document": "def unwatch(models=None):\n \"\"\"Remove pytorch model topology, gradient and parameter hooks.\n\n Args:\n models: (list) Optional list of pytorch models that have had watch called on them\n \"\"\"\n if models:\n if not isinstance(models, (tuple, list)):\n models = (models,)\n for model in models:\n if not hasattr(model, \"_wandb_hook_names\"):\n wandb.termwarn(\"%s model has not been watched\" % model)\n else:\n for name in model._wandb_hook_names:\n wandb.run._torch.unhook(name)\n delattr(model, \"_wandb_hook_names\")\n # TODO: we should also remove recursively model._wandb_watch_called\n\n else:\n wandb.run._torch.unhook_all()", "matching_document_document_id": "82e6e3a9-d9f0-4a47-b3ef-66fbe33dc419", "query_weight": 0.6038709677419355 }, { "query_id": "a26e2297-5488-4cdd-92b2-c7e9b1b283e9", "query": "Is there a specific setting or parameter in Weights & Biases to manage run resumption and checkpointing effectively?", "matching_document": "## Resume Runs\n#### Resume Guidance\n1. As described above, runs can be resumed by passing`resume=True` to `wandb.init()`. This can be thought of as auto-resuming, where we \u201cautomatically\u201d pick up from where an aborted run left off. If your process doesn't exit successfully, the next time you run it W&B will start logging from the last step.\n2. Note: This only works if you are running your script in the same directory as the one that failed as the file is stored at: `wandb/wandb-resume.json`.\n3. The other form of resume requires you to provide the actual run id: `wandb.init(id=run_id)` and then when you resume (if you want to be sure that it is resuming, you do `wandb.init(id=run_id, resume=\"must\")`.\n4. You can also have full control over resuming if you manage the `run_id`. We provide a utility to generate `run_id`: `wandb.util.generate_id()`. As long as you set the id to one of these unique ids for each unique run, you can say `resume=\"allow\"` and W&B will automatically resume the run with that id.More context regarding automatic and controlled resuming can be found in this section.\n2. `wandb.restore`\n\\* This will allow you to log new historical values for your metrics to a run starting from where you left off but does not take care of re-establishing the state of your code, you will need to make sure you have written checkpoints that you can load!\n\\* You can use `wandb.save` to record the state of your run via checkpoint files. Create a checkpoint file through `wandb.save()`, which can then be used through `wandb.init(resume=)`. This report illustrates how to save and restore models with W&B.", "matching_document_document_id": "3effe167-849e-453c-a198-3e826eb7e84a", "query_weight": 0.5574193548387096 }, { "query_id": "cf147c30-5110-4573-9fd0-76a41e3c7740", "query": "resume training from last epoch", "matching_document": "## GCP with Weights & Biases\n### Advanced Features: Resuming and Grouping Runs\n#### Resuming Runs\nNow, let\u2019s say, while training my model I accidentally restarted my Jupyter Notebook instance. After I restarted the kernel, to be able to resume the training from exactly where it was last, I would do the following - \n\n```\nmodel = tf.keras.models.load_model(wandb.restore(\"model-best.h5\").name)\n``` \n\nIt is important to note this will only work if you supply the WandbCallback while calling model.fit(). \n\nAfter loading the model, I simply need to compile it in exactly the same way I did before restarting the kernel and I would call model.fit() then - \n\n```\nmodel.compile(loss='categorical_crossentropy', optimizer=sgd, metrics=\n['accuracy'])\nmodel.fit(X_train, y_train, validation_data=(X_test, y_test),\nepochs=20, initial_epoch=wandb.run.step,\ncallbacks=[WandbCallback(data_type=\"image\", labels=labels,\nsave_model=True)])\n``` \n\nPay attention to the initial_epoch argument. The training will begin from exactly where it was left off.", "matching_document_document_id": "bdefcfdb-0da7-409f-ab64-7e8e1c245cd0", "query_weight": 0.5574193548387096 }, { "query_id": "41da350d-599a-4afa-94da-0839a1c5ef52", "query": "I need some help with run resuming, does it automatically use the last saved checkpoint artifact model? How do i get my run resuming to start at the same place it ended? Right now its starting from epoch 1 every time.", "matching_document": "## Resume Runs\n#### Resume Guidance\n1. As described above, runs can be resumed by passing`resume=True` to `wandb.init()`. This can be thought of as auto-resuming, where we \u201cautomatically\u201d pick up from where an aborted run left off. If your process doesn't exit successfully, the next time you run it W&B will start logging from the last step.\n2. Note: This only works if you are running your script in the same directory as the one that failed as the file is stored at: `wandb/wandb-resume.json`.\n3. The other form of resume requires you to provide the actual run id: `wandb.init(id=run_id)` and then when you resume (if you want to be sure that it is resuming, you do `wandb.init(id=run_id, resume=\"must\")`.\n4. You can also have full control over resuming if you manage the `run_id`. We provide a utility to generate `run_id`: `wandb.util.generate_id()`. As long as you set the id to one of these unique ids for each unique run, you can say `resume=\"allow\"` and W&B will automatically resume the run with that id.More context regarding automatic and controlled resuming can be found in this section.\n2. `wandb.restore`\n\\* This will allow you to log new historical values for your metrics to a run starting from where you left off but does not take care of re-establishing the state of your code, you will need to make sure you have written checkpoints that you can load!\n\\* You can use `wandb.save` to record the state of your run via checkpoint files. Create a checkpoint file through `wandb.save()`, which can then be used through `wandb.init(resume=)`. This report illustrates how to save and restore models with W&B.", "matching_document_document_id": "3effe167-849e-453c-a198-3e826eb7e84a", "query_weight": 0.5574193548387096 }, { "query_id": "48e68db7-3249-4d31-8023-94f9d6ab464d", "query": "Best practices for checkpointing and run resumption in machine learning experiments", "matching_document": "## Resume Runs\n#### Resume Guidance\n1. As described above, runs can be resumed by passing`resume=True` to `wandb.init()`. This can be thought of as auto-resuming, where we \u201cautomatically\u201d pick up from where an aborted run left off. If your process doesn't exit successfully, the next time you run it W&B will start logging from the last step.\n2. Note: This only works if you are running your script in the same directory as the one that failed as the file is stored at: `wandb/wandb-resume.json`.\n3. The other form of resume requires you to provide the actual run id: `wandb.init(id=run_id)` and then when you resume (if you want to be sure that it is resuming, you do `wandb.init(id=run_id, resume=\"must\")`.\n4. You can also have full control over resuming if you manage the `run_id`. We provide a utility to generate `run_id`: `wandb.util.generate_id()`. As long as you set the id to one of these unique ids for each unique run, you can say `resume=\"allow\"` and W&B will automatically resume the run with that id.More context regarding automatic and controlled resuming can be found in this section.\n2. `wandb.restore`\n\\* This will allow you to log new historical values for your metrics to a run starting from where you left off but does not take care of re-establishing the state of your code, you will need to make sure you have written checkpoints that you can load!\n\\* You can use `wandb.save` to record the state of your run via checkpoint files. Create a checkpoint file through `wandb.save()`, which can then be used through `wandb.init(resume=)`. This report illustrates how to save and restore models with W&B.", "matching_document_document_id": "3effe167-849e-453c-a198-3e826eb7e84a", "query_weight": 0.5574193548387096 }, { "query_id": "3e2659d7-46ee-4bee-8521-a7add3d5b04f", "query": "Are there any settings to adjust for smoother wandb table interaction?", "matching_document": "## Log Tables\n### Save Tables\nAfter you generate a table of data in your script, for example a table of model predictions, save it to W&B to visualize the results live. \n\n### Log a table to a run \n\nUse `wandb.log()` to save your table to the run, like so: \n\n```\nrun = wandb.init()\nmy\\_table = wandb.Table(columns=[\"a\", \"b\"], data=[[\"1a\", \"1b\"], [\"2a\", \"2b\"]])\nrun.log({\"table\\_key\": my\\_table})\n\n``` \n\nEach time a table is logged to the same key, a new version of the table is created and stored in the backend. This means you can log the same table across multiple training steps to see how model predictions improve over time, or compare tables across different runs, as long as they're logged to the same key. You can log up to 200,000 rows. \n\n:::info\nTo log more than 200,000 rows, you can override the limit with: \n\n`wandb.Table.MAX_ARTIFACTS_ROWS = X` \n\nHowever, this would likely cause performance issues, such as slower queries, in the UI.\n::: \n\n### Access tables programmatically", "matching_document_document_id": "17f05ffb-3eda-4203-b9c3-9cba9b6bd8ee", "query_weight": 0.9754838709677419 }, { "query_id": "2ad2db58-00c5-48b4-9d32-b4f260821b2e", "query": "wandb table performance optimization techniques", "matching_document": "## Limits & Performance\n\nExperiments Limits and Performance \n\n## Best Practices for Fast Pages \n\nKeep your pages in W&B faster and more responsive by logging within these suggested bounds. \n\n### Logged Metrics \n\nUse `wandb.log` to track experiment metrics. Once logged, these metrics generate charts and show up in tables. Too much logged data can make the app slow. \n\n#### Count of Distinct Metrics \n\nKeep the total number of distinct metrics under 10,000. \n\n```\nwandb.log(\n{\n\"a\": 1, # \"a\" is a distinct metric\n\"b\": {\n\"c\": \"hello\", # \"b.c\" is a distinct metric\n\"d\": [1, 2, 3], # \"b.d\" is a distinct metric\n},\n}\n) # 3 distinct metrics logged\n\n``` \n\n:::caution\nWe automatically flatten nested values, so if you pass us a dictionary we will turn it into a dot-separated name. For config values, we support 3 dots in the name. For summary values, we support 4 dots.\n::: \n\nLog related media to the same metric name: \n\n#### Value Width \n\n#### Metric Frequency \n\n#### Config Size \n\n#### Run Count", "matching_document_document_id": "24f178f5-84f3-4abc-ab27-a627c1e9ccdd", "query_weight": 0.9754838709677419 }, { "query_id": "fe4fd1f9-0e7b-40d3-8349-639f71c521be", "query": "How to optimize wandb table loading times?", "matching_document": "## Limits & Performance\n\nExperiments Limits and Performance \n\n## Best Practices for Fast Pages \n\nKeep your pages in W&B faster and more responsive by logging within these suggested bounds. \n\n### Logged Metrics \n\nUse `wandb.log` to track experiment metrics. Once logged, these metrics generate charts and show up in tables. Too much logged data can make the app slow. \n\n#### Count of Distinct Metrics \n\nKeep the total number of distinct metrics under 10,000. \n\n```\nwandb.log(\n{\n\"a\": 1, # \"a\" is a distinct metric\n\"b\": {\n\"c\": \"hello\", # \"b.c\" is a distinct metric\n\"d\": [1, 2, 3], # \"b.d\" is a distinct metric\n},\n}\n) # 3 distinct metrics logged\n\n``` \n\n:::caution\nWe automatically flatten nested values, so if you pass us a dictionary we will turn it into a dot-separated name. For config values, we support 3 dots in the name. For summary values, we support 4 dots.\n::: \n\nLog related media to the same metric name: \n\n#### Value Width \n\n#### Metric Frequency \n\n#### Config Size \n\n#### Run Count", "matching_document_document_id": "24f178f5-84f3-4abc-ab27-a627c1e9ccdd", "query_weight": 0.9754838709677419 }, { "query_id": "f23ed53c-d335-4306-994a-ce1f500ed078", "query": "Improving wandb table performance and responsiveness", "matching_document": "## Limits & Performance\n\nExperiments Limits and Performance \n\n## Best Practices for Fast Pages \n\nKeep your pages in W&B faster and more responsive by logging within these suggested bounds. \n\n### Logged Metrics \n\nUse `wandb.log` to track experiment metrics. Once logged, these metrics generate charts and show up in tables. Too much logged data can make the app slow. \n\n#### Count of Distinct Metrics \n\nKeep the total number of distinct metrics under 10,000. \n\n```\nwandb.log(\n{\n\"a\": 1, # \"a\" is a distinct metric\n\"b\": {\n\"c\": \"hello\", # \"b.c\" is a distinct metric\n\"d\": [1, 2, 3], # \"b.d\" is a distinct metric\n},\n}\n) # 3 distinct metrics logged\n\n``` \n\n:::caution\nWe automatically flatten nested values, so if you pass us a dictionary we will turn it into a dot-separated name. For config values, we support 3 dots in the name. For summary values, we support 4 dots.\n::: \n\nLog related media to the same metric name: \n\n#### Value Width \n\n#### Metric Frequency \n\n#### Config Size \n\n#### Run Count", "matching_document_document_id": "24f178f5-84f3-4abc-ab27-a627c1e9ccdd", "query_weight": 0.9754838709677419 }, { "query_id": "9fcb6444-4139-4834-b51e-c7a899aa2922", "query": "What are accelerate launch arguments?", "matching_document": "## Launching Distributed Code\nHuggingFace Accelerate has a special CLI command to help you launch your code in your system through accelerate launch. This command wraps around all of the different commands needed to launch your script on various platforms without you having to remember what each of them is. \n\nYou can launch your script quickly by using: \n\n```\naccelerate launch {script_name.py} --arg1 --arg2 ...\n``` \n\nJust put accelerate launch at the start of your command, and pass in additional arguments and parameters to your script afterward like normal! \n\nSince this runs the various torch spawn methods, all of the expected environment variables can be modified here as well. For example, here is how to use accelerate launch with a single GPU: \n\n```\nCUDA_VISIBLE_DEVICES=\"0\" accelerate launch {script_name.py} --arg1 --arg2 ...\n``` \n\nTo explore more options, you can take a look at the documentation [here](https://huggingface.co/docs/accelerate/v0.13.2/en/basic_tutorials/launch).", "matching_document_document_id": "98ab6d0b-2a8a-42d4-9393-4fe0cec531d9", "query_weight": 0.6503225806451612 }, { "query_id": "a121a4d3-98e7-426d-8c6a-a1bc7700ee34", "query": "How to track notes or metadata in accelerate?", "matching_document": "## Experiment Tracking with Weights & Biases\n* : The name of the project. This would be passed into your project argument behind the hood.\n* : configuration to be logged. Passed into 's config argument behind the hood.\n* : A nested dictionary of kwargs to be passed to a specific tracker\u2019s `__init__` function. You can pass any other argument that takes as a key-value pair in this argument. \n\nHere's an example of how you can initialize a W&B run using HuggingFace Accelerate. \n\n```\nfrom accelerate import Accelerator\n\naccelerator = Accelerator()\nhps = {\"num_epochs\": 5, \"learning_rate\": 1e-4, \"batch_size\": 16}\naccelerator.init_trackers(\n\"my_project\",\nconfig=hps,\ninit_kwargs={\n\"wandb\": {\n\"notes\": \"testing accelerate pipeline\",\n\"tags\": [\"tag_a\", \"tag_b\"],\n\"entity\": \"gladiator\",\n}\n},\n)\n```", "matching_document_document_id": "e610f0f9-6bc3-40fb-ab8b-ee79c717aab5", "query_weight": 0.6503225806451612 }, { "query_id": "90104271-440f-4706-a4e7-74849510f155", "query": "i am using init_kwargs with accelerator for main process in my training network script. how can i track notes in the accelerate launch args?", "matching_document": "## Experiment Tracking with Weights & Biases\n* : The name of the project. This would be passed into your project argument behind the hood.\n* : configuration to be logged. Passed into 's config argument behind the hood.\n* : A nested dictionary of kwargs to be passed to a specific tracker\u2019s `__init__` function. You can pass any other argument that takes as a key-value pair in this argument. \n\nHere's an example of how you can initialize a W&B run using HuggingFace Accelerate. \n\n```\nfrom accelerate import Accelerator\n\naccelerator = Accelerator()\nhps = {\"num_epochs\": 5, \"learning_rate\": 1e-4, \"batch_size\": 16}\naccelerator.init_trackers(\n\"my_project\",\nconfig=hps,\ninit_kwargs={\n\"wandb\": {\n\"notes\": \"testing accelerate pipeline\",\n\"tags\": [\"tag_a\", \"tag_b\"],\n\"entity\": \"gladiator\",\n}\n},\n)\n```", "matching_document_document_id": "e610f0f9-6bc3-40fb-ab8b-ee79c717aab5", "query_weight": 0.6503225806451612 }, { "query_id": "b6b7f01c-f910-4ded-bc0b-5c869e43b33e", "query": "Methods to track notes in accelerate launch arguments.", "matching_document": "## Experiment Tracking with Weights & Biases\n* : The name of the project. This would be passed into your project argument behind the hood.\n* : configuration to be logged. Passed into 's config argument behind the hood.\n* : A nested dictionary of kwargs to be passed to a specific tracker\u2019s `__init__` function. You can pass any other argument that takes as a key-value pair in this argument. \n\nHere's an example of how you can initialize a W&B run using HuggingFace Accelerate. \n\n```\nfrom accelerate import Accelerator\n\naccelerator = Accelerator()\nhps = {\"num_epochs\": 5, \"learning_rate\": 1e-4, \"batch_size\": 16}\naccelerator.init_trackers(\n\"my_project\",\nconfig=hps,\ninit_kwargs={\n\"wandb\": {\n\"notes\": \"testing accelerate pipeline\",\n\"tags\": [\"tag_a\", \"tag_b\"],\n\"entity\": \"gladiator\",\n}\n},\n)\n```", "matching_document_document_id": "e610f0f9-6bc3-40fb-ab8b-ee79c717aab5", "query_weight": 0.6503225806451612 }, { "query_id": "f03e4a2a-2637-4765-8472-41411cba2f91", "query": "How to integrate wandb with accelerate for passing launch arguments?", "matching_document": "## Hugging Face Accelerate\n### Start logging with Accelerate\n```\nfrom accelerate import Accelerator\n\n# Tell the Accelerator object to log with wandb\naccelerator = Accelerator(log\\_with=\"wandb\")\n\n# Initialise your wandb run, passing wandb parameters and any config information\naccelerator.init\\_trackers(\nproject\\_name=\"my\\_project\",\nconfig={\"dropout\": 0.1, \"learning\\_rate\": 1e-2}\ninit\\_kwargs={\"wandb\": {\"entity\": \"my-wandb-team\"}}\n)\n\n...\n\n# Log to wandb by calling `accelerator.log`, `step` is optional\naccelerator.log({\"train\\_loss\": 1.12, \"valid\\_loss\": 0.8}, step=global\\_step)\n\n# Make sure that the wandb tracker finishes correctly\naccelerator.end\\_training()\n\n```", "matching_document_document_id": "45d887a0-2149-431d-9e85-eab8ddfeb6f0", "query_weight": 0.6503225806451612 }, { "query_id": "217e51ca-aaf4-4c58-a7ea-a31fc0cfaa76", "query": "Is it possible to log gradients using Weights & Biases in TensorFlow without WandbCallback?", "matching_document": "if self.log_gradients:\n if int(tf.__version__.split(\".\")[0]) < 2:\n raise Exception(\"Gradient logging requires tensorflow 2.0 or higher.\")\n if self.training_data is None:\n raise ValueError(\n \"training_data argument is required for gradient logging.\"\n )\n if isinstance(self.training_data, (list, tuple)):\n if len(self.training_data) != 2:\n raise ValueError(\"training data must be a tuple of length two\")\n self._training_data_x, self._training_data_y = self.training_data\n else:\n self._training_data_x = (\n self.training_data\n ) # generator, tf.data.Dataset etc\n self._training_data_y = None\n\n # From Keras\n if mode not in [\"auto\", \"min\", \"max\"]:\n print(f\"WandbCallback mode {mode} is unknown, fallback to auto mode.\")\n mode = \"auto\"", "matching_document_document_id": "9e7ef2fe-91c2-465a-b2a2-2f12ff646f01", "query_weight": 1.161290322580645 }, { "query_id": "c9357a5c-7110-47be-bb98-2348c8b49572", "query": "llama model gradients and activations", "matching_document": "## Training Loop\n```\nfrom types import SimpleNamespace\n\ngradient_accumulation_steps = 32 // batch_size\n\nconfig = SimpleNamespace(\nmodel_id='meta-llama/Llama-2-7b-hf',\ndataset_name=\"alpaca-gpt4\",\nprecision=\"bf16\", # faster and better than fp16, requires new GPUs\nn_freeze=24, # How many layers we don't train, LLama 7B has 32.\nlr=2e-4,\nn_eval_samples=10, # How many samples to generate on validation\nmax_seq_len=max_sequence_len, # Length of the sequences to pack\nepochs=3, # we do 3 pasess over the dataset.\ngradient_accumulation_steps=gradient_accumulation_steps, # evey how many iterations we update the gradients, simulates larger batch sizes\nbatch_size=batch_size, # what my GPU can handle, depends on how many layers are we training\nlog_model=False, # upload the model to W&B?\nmom=0.9, # optim param\ngradient_checkpointing = True, # saves even more memory\nfreeze_embed = True, # why train this? let's keep them frozen \u2744\ufe0f\n)\n\nconfig.total_train_steps = config.epochs * len(train_dataloader) // config.gradient_accumulation_steps\n```", "matching_document_document_id": "177bb79c-4d16-4a27-9ec0-4726f55c6b1c", "query_weight": 1.161290322580645 }, { "query_id": "e98b4f2c-169c-4d51-9558-1f557f9a368b", "query": "Examples of GradientTape with Weights & Biases", "matching_document": "## Customizing Training Loops in TensorFlow 2.0\n### Model training and the use of W&B\nThe model is already defined for us. The instructions on how to train the model are defined the following function which we revisited earlier: \n\n```\n# Train the model\n@tf.function\ndef model_train(features, labels):\n# Define the GradientTape context\nwith tf.GradientTape() as tape:\n# Get the probabilities\npredictions = model(features)\n# Calculate the loss\nloss = loss_func(labels, predictions)\n# Get the gradients\ngradients = tape.gradient(loss, model.trainable_variables)\n# Update the weights\noptimizer.apply_gradients(zip(gradients, model.trainable_variables))\n\n# Update the loss and accuracy\ntrain_loss(loss)\ntrain_acc(labels, predictions)\n``` \n\nSimilarly, we can define a little function to evaluate our model: \n\n```\n# Validating the model\n@tf.function\ndef model_validate(features, labels):\npredictions = model(features)\nv_loss = loss_func(labels, predictions)\n\nvalid_loss(v_loss)\nvalid_acc(labels, predictions)\n``` \n\nNow, we are absolutely ready to start the model training:", "matching_document_document_id": "4cc24b08-a610-454b-9c04-b3a92bebcfe4", "query_weight": 1.161290322580645 }, { "query_id": "dd0badd6-4c82-41cd-a355-075777bb05df", "query": "Examples of logging gradients in TensorFlow using Weights & Biases.", "matching_document": "## \ud83c\udf3e Conclusion\n\nWeights and Biases's Keras integration enables experiment tracking and plenty more with just few lines of code. In this notebook, we've seen some advanced usage of Keras WandbCallback and different ways of using W&B Tables for evaluation and data exploration. \n\nTo sum up all you need is a free W&B account, import the WandbCallback and pass it to model.fit(callbacks=[.]) just like any callback. There are a few more arguments that you can learn about in the [documentation page here](https://docs.wandb.ai/ref/python/integrations/keras/wandbcallback). Of particular note: \n\n* You can log the metrics for each batch by setting `log_batch_frequency=1`\n* You can log the gradients of each layer to debug vanishing or exploding gradient issue by setting `log_gradients=True`. You will also have to provide the `training_data` in the format of `(X, y)`.\n* if your task is semantic segmentation you can set `input_type=segmentation_mask`.", "matching_document_document_id": "4f0027f6-ce5d-44c1-827b-4d969f2ef301", "query_weight": 1.161290322580645 }, { "query_id": "bcaac1df-a19e-4fc5-b86d-fe1640c9a9bb", "query": "How to correctly implement a TensorFlow custom callback for logging gradients?", "matching_document": "### WandbCallback [Legacy]\n| Arguments | |\n| --- | --- |\n| `monitor` | (str) name of metric to monitor. Defaults to `val_loss`. |\n| `mode` | (str) one of {`auto`, `min`, `max`}. `min` - save model when monitor is minimized `max` - save model when monitor is maximized `auto` - try to guess when to save the model (default). |\n| `save_model` | True - save a model when monitor beats all previous epochs False - don't save models |\n| `save_graph` | (boolean) if True save model graph to wandb (default to True). |\n| `save_weights_only` | (boolean) if True, then only the model's weights will be saved (`model.save_weights(filepath)`), else the full model is saved (`model.save(filepath)`). |\n| `log_weights` | (boolean) if True save histograms of the model's layer's weights. |\n| `log_gradients` | (boolean) if True log histograms of the training gradients |\n| `training_data` | (tuple) Same format `(X,y)` as passed to `model.fit`. This is needed for calculating gradients - this is mandatory if `log_gradients` is `True`. |\n| `validation_data` | (tuple) Same format `(X,y)` as passed to `model.fit`. A set of data for wandb to visualize. If this is set, every epoch, wandb will make a small number of predictions and save the results for later visualization. |\n| `generator` | (generator) a generator that returns validation data for wandb to visualize. This generator should return tuples `(X,y)`. Either `validate_data` or generator should be set for wandb to visualize specific data examples. |\n| `validation_steps` | (int) if `validation_data` is a generator, how many steps to run the generator for the full validation set. |", "matching_document_document_id": "074f0d86-f39c-46f1-a79e-9eacfeb004a8", "query_weight": 1.161290322580645 }, { "query_id": "1adc6ac4-9f7e-4f90-b826-346dee827128", "query": "optimizing memory usage", "matching_document": "## Monitor & Improve GPU Usage for Model Training\n#### 1. Measure your GPU usage consistently over your entire training runs\nYou can\u2019t improve GPU usage without measuring it. It\u2019s not hard to take a snapshot of your usage with useful tools like nvidia-smi, but a simple way to find issues is to track usage over time. Anyone can turn on system monitoring in the background, which will track GPU, CPU, memory usage etc over time by adding two lines to their code: \n\n```\nimport wandb\nwandb.init()\n``` \n\nThe wandb.init() function will create a lightweight child process that will collect system metrics and send them to a wandb server where you can look at them and compare across runs with graphs like these: \n\nThe danger of taking a single measurement is that GPU usage can change over time. This is a common pattern we see where our user Boris is training an RNN; mid-training, his usage plummets from 80 percent to around 25 percent.", "matching_document_document_id": "f57cb60e-1cde-4855-a5e2-1688ac4755e5", "query_weight": 2.0903225806451613 }, { "query_id": "3ed4c298-5afc-4633-8ad4-a1b125223290", "query": "Audio file processing", "matching_document": "## An Example of Audio Classification Using Keras\n### Step 3: Preprocessing the Data Set\nNext, we load the audio data set and the CSV data set. We then iterate through each row of the metadata and use the information in each to locate the corresponding audio file in the data_path directory. We load the audio file using librosa.load() and resample it to a target sampling rate of 22050 Hz. \n\n```\ndef load_data(data_path, metadata_path):\nfeatures = []\nlabels = []\n\nmetadata = pd.read_csv(metadata_path)\n\nfor index, row in metadata.iterrows():\nfile_path = os.path.join(data_path, f\"fold{row['fold']}\", f\"{row['slice_file_name']}\")\n\n# Load the audio file and resample it\ntarget_sr = 22050\naudio, sample_rate = librosa.load(file_path, sr=target_sr)\n\n# Extract MFCC features\nmfccs = librosa.feature.mfcc(y=audio, sr=target_sr, n_mfcc=40)\nmfccs_scaled = np.mean(mfccs.T, axis=0)\n\n# Append features and labels\nfeatures.append(mfccs_scaled)\nlabels.append(row['class'])\n\nreturn np.array(features), np.array(labels)\n```", "matching_document_document_id": "a454c725-770b-46aa-93b9-ca6105b0529c", "query_weight": 2.0903225806451613 }, { "query_id": "a858dc45-6edd-4606-85d7-7f035ee7285d", "query": "best practices for managing large datasets in machine learning experiments", "matching_document": "## Storage\n\nIf you are approaching or exceeding your storage limit, there are multiple paths forward to manage your data. The path that's best for you will depend on your account type and your current project setup. \n\n## Manage storage consumption \n\nW&B offers different methods of optimizing your storage consumption: \n\n* Use\u00a0reference artifacts\u00a0to track files saved outside the W&B system, instead of uploading them to W&B storage.\n* Use an external cloud storage bucket for storage. *(Enterprise only)* \n\n## Delete data \n\nYou can also choose to delete data to remain under your storage limit. There are several ways to do this: \n\n* Delete data interactively with the app UI.\n* Set a TTL policy on Artifacts so they are automatically deleted.", "matching_document_document_id": "6dea2d7d-58f2-4830-8dc1-27b3bf719859", "query_weight": 2.0903225806451613 }, { "query_id": "29162817-2639-4ea0-826c-ce657edf92e5", "query": "How to pass model name as metadata using wandb?", "matching_document": "## Add wandb to Any Library\n#### Defining A wandb Run Config\n\nWith a `wandb` run config you can provide metadata about your model, dataset, and so on when you create a W&B Run. You can use this information to compare different experiments and quickly understand what are the main differences. \n\nTypical config parameters you can log include: \n\n* Model name, version, architecture parameters etc\n* Dataset name, version, number of train/val examples etc\n* Training parameters such as learning rate, batch size, optimizer etc \n\nThe following code snippet shows how to log a config: \n\n```\nconfig = {\u201cbatch\\_size\u201d:32, \u2026}\nwandb.init(\u2026, config=config)\n\n``` \n\n#### Updating The wandb config \n\nUse `wandb.config.update` to update the config. Updating your configuration dictionary is useful when parameters are obtained after the dictionary was defined, for example you might want to add a model\u2019s parameters after the model is instantiated. \n\n```\nwandb.config.update({\u201cmodel\\_parameters\u201d = 3500})\n\n```", "matching_document_document_id": "3fc1a9c7-c2c8-4a8a-89b8-1bf550a374dd", "query_weight": 0.5574193548387096 }, { "query_id": "1bba9bed-1b74-4dec-b7b0-802ef49502e5", "query": "pass model name", "matching_document": "## Log Models\n### Log a model to a W&B run\nOptionally provide a name for the model artifact for the `name` parameter. If `name` is not specified, W&B will use the basename of the input path prepended with the run ID as the name. \n\n:::tip\nKeep track of the `name` that you, or W&B assigns, to the model. You will need the name of the model to retrieve the model path with the `use_model` method.\n::: \n\nSee `log_model` in the API Reference guide for more information on possible parameters. \n\nExample: Log a model to a run", "matching_document_document_id": "20bfd336-abd3-4dce-86ca-85ce88ac045b", "query_weight": 0.5574193548387096 }, { "query_id": "a5fb43db-e762-4fc0-b37e-95ba245732b7", "query": "please show me how to log training output_name model of the current training run", "matching_document": "### Checkpoing your model weights ###\nx = torch.randn(1, 3, 224, 224, requires_grad=True)\ntorch.onnx.export(model, # model being run\nx, # model input (or a tuple for multiple inputs)\n\"model.onnx\", # where to save the model (can be a file or file-like object)\nexport_params=True, # store the trained parameter weights inside the model file\nopset_version=10, # the ONNX version to export the model to\ndo_constant_folding=True, # whether to execute constant folding for optimization\ninput_names = ['input'], # the model's input names\noutput_names = ['output'], # the model's output names\ndynamic_axes={'input' : {0 : 'batch_size'}, # variable length axes\n'output' : {0 : 'batch_size'}})\n\nart = wandb.Artifact(f\"anomaly-classifier-{wandb.run.id}\",\ntype=\"model\",\nmetadata={'format': 'onnx',\n'num_classes': len(nature_module.nature_dataset.class_names),\n'model_type': wandb.config['model_type'],\n'model_input_size': wandb.config['input_size'],\n'index_to_class': nature_module.nature_dataset.idx_to_class})\n\nart.add_file(\"model.onnx\")\n\n### Add aliases to keep track of your best checkpoints over time\nwandb.log_artifact(art, aliases=[\"best\", \"latest\"] if is_best else None)\nif is_best:\nbest_model = art\n\n```", "matching_document_document_id": "217130bb-db79-4a71-8581-923991613aa1", "query_weight": 0.5574193548387096 }, { "query_id": "54f62126-7bcd-46f9-8e09-367867e38d8a", "query": "pass model name as metadata", "matching_document": "## Artifact\n| Arguments | |\n| --- | --- |\n| `name` | A human-readable name for this artifact, which is how you can identify this artifact in the UI or reference it in `use_artifact` calls. Names can contain letters, numbers, underscores, hyphens, and dots. The name must be unique across a project. |\n| `type` | The type of the artifact, which is used to organize and differentiate artifacts. Common types include `dataset` or `model`, but you can use any string containing letters, numbers, underscores, hyphens, and dots. |\n| `description` | Free text that offers a description of the artifact. The description is markdown rendered in the UI, so this is a good place to place tables, links, etc. |\n| `metadata` | Structured data associated with the artifact, for example class distribution of a dataset. This will eventually be queryable and plottable in the UI. There is a hard limit of 100 total keys. | \n\n| Returns | |\n| --- | --- |\n| An `Artifact` object. | | \n\n#### Examples: \n\nBasic usage: \n\n## Methods \n\n### `add`", "matching_document_document_id": "922eb81a-e3ae-407c-968d-c36695150581", "query_weight": 0.5574193548387096 }, { "query_id": "7129171d-4690-4461-b210-43e8a68b2ccf", "query": "Examples of using add_named_result with WandTracer", "matching_document": "## Non-Lang Chain Implementation\n```\ntool\\_span.add\\_named\\_result(\n{\"input\": \"search: google founded in year\"},\n{\"response\": \"1998\"}\n)\nchain\\_span.add\\_named\\_result(\n{\"input\": \"calculate: 2023 - 1998\"},\n{\"response\": \"25\"}\n)\nllm\\_span.add\\_named\\_result(\n{\"input\": \"calculate: 2023 - 1998\", \"system\": \"you are a helpful assistant\", },\n{\"response\": \"25\", \"tokens\\_used\":218}\n)\n\nparent\\_span.add\\_child\\_span(tool\\_span)\nparent\\_span.add\\_child\\_span(chain\\_span)\n\nparent\\_span.add\\_named\\_result({\"user\": \"calculate: 2023 - 1998\"},\n{\"response\": \"25 years old\"})\n\n``` \n\nYou can then log the parent\\_span to W&B like as below. \n\n```\nrun = wandb.init(name=\"manual\\_span\\_demo\", project=\"wandb\\_prompts\\_demo\")\nrun.log({\"trace\": trace\\_tree.WBTraceTree(parent\\_span)})\nrun.finish()\n\n``` \n\nClicking on the W&B Run link generated will take you to a workspace where you can inspect the Trace created.", "matching_document_document_id": "7bde4779-6366-4502-af8f-2e8e25223f20", "query_weight": 0.7432258064516128 }, { "query_id": "d43e2ab2-2337-4627-9ae2-ef9524428826", "query": "Documentation for WandTracer in Weights & Biases", "matching_document": "## WandbTracer\n\nView source on GitHub \n\nCallback Handler that logs to Weights and Biases. \n\n```\nWandbTracer() -> Any\n\n``` \n\nThis handler will log the model architecture and run traces to Weights and Biases. This will ensure that all LangChain activity is logged to W&B. \n\n| Attributes | |\n| --- | --- |\n| `always_verbose` | Whether to call verbose callbacks even if verbose is False. |\n| `ignore_agent` | Whether to ignore agent callbacks. |\n| `ignore_chain` | Whether to ignore chain callbacks. |\n| `ignore_llm` | Whether to ignore LLM callbacks. | \n\n## Methods \n\n### `finish` \n\nView source \n\n```\n@staticmethod\nfinish() -> None\n\n``` \n\nWaits for all asynchronous processes to finish and data to upload. \n\n### `finish_run` \n\nView source \n\n```\nfinish\\_run() -> None\n\n``` \n\nWaits for W&B data to upload. \n\n### `init` \n\nView source \n\n### `init_run` \n\nView source \n\n```python\ninit_run(\nrun_args: Optional[WandbRunArgs] = None\n) -> None\n\n``` \n\nInitialize wandb if it has not been initialized. \n\n### `on_text`", "matching_document_document_id": "26adf871-4ee0-4fa2-8207-dde9db3639d7", "query_weight": 0.7432258064516128 }, { "query_id": "467307a4-e962-434c-b8d0-164a40e212cf", "query": "trace tree Weights & Biases", "matching_document": "\"\"\"Media object for trace tree data.\n\n Arguments:\n root_span (Span): The root span of the trace tree.\n model_dict (dict, optional): A dictionary containing the model dump.\n NOTE: model_dict is a completely-user-defined dict. The UI will render\n a JSON viewer for this dict, giving special treatment to dictionaries\n with a `_kind` key. This is because model vendors have such different\n serialization formats that we need to be flexible here.\n \"\"\"", "matching_document_document_id": "dbf0195c-df1e-4517-a0ad-e7a097335d3f", "query_weight": 0.7432258064516128 }, { "query_id": "bb782b76-623e-4c2c-ad34-442b39bd698e", "query": "Can you provide an example of using add_named_result with a trace tree in Weights & Biases?", "matching_document": "## Non-Lang Chain Implementation\n```\ntool\\_span.add\\_named\\_result(\n{\"input\": \"search: google founded in year\"},\n{\"response\": \"1998\"}\n)\nchain\\_span.add\\_named\\_result(\n{\"input\": \"calculate: 2023 - 1998\"},\n{\"response\": \"25\"}\n)\nllm\\_span.add\\_named\\_result(\n{\"input\": \"calculate: 2023 - 1998\", \"system\": \"you are a helpful assistant\", },\n{\"response\": \"25\", \"tokens\\_used\":218}\n)\n\nparent\\_span.add\\_child\\_span(tool\\_span)\nparent\\_span.add\\_child\\_span(chain\\_span)\n\nparent\\_span.add\\_named\\_result({\"user\": \"calculate: 2023 - 1998\"},\n{\"response\": \"25 years old\"})\n\n``` \n\nYou can then log the parent\\_span to W&B like as below. \n\n```\nrun = wandb.init(name=\"manual\\_span\\_demo\", project=\"wandb\\_prompts\\_demo\")\nrun.log({\"trace\": trace\\_tree.WBTraceTree(parent\\_span)})\nrun.finish()\n\n``` \n\nClicking on the W&B Run link generated will take you to a workspace where you can inspect the Trace created.", "matching_document_document_id": "7bde4779-6366-4502-af8f-2e8e25223f20", "query_weight": 0.7432258064516128 }, { "query_id": "69ecea1a-9d22-4cd2-8aff-5b41931298aa", "query": "fix wandb.sdk.service.service.ServiceStartTimeoutError", "matching_document": "def _wait_for_ports(\n self, fname: str, proc: Optional[subprocess.Popen] = None\n ) -> None:\n \"\"\"Wait for the service to write the port file and then read it.\n\n Args:\n fname: The path to the port file.\n proc: The process to wait for.\n\n Raises:\n ServiceStartTimeoutError: If the service takes too long to start.\n ServiceStartPortError: If the service writes an invalid port file or unable to read it.\n ServiceStartProcessError: If the service process exits unexpectedly.", "matching_document_document_id": "7b214499-9701-4d4e-95ff-34635c3b816e", "query_weight": 1.207741935483871 }, { "query_id": "ccc15293-9603-4803-8ba6-505525d89b20", "query": "wandb service start timeout solutions", "matching_document": "def _wait_for_ports(\n self, fname: str, proc: Optional[subprocess.Popen] = None\n ) -> None:\n \"\"\"Wait for the service to write the port file and then read it.\n\n Args:\n fname: The path to the port file.\n proc: The process to wait for.\n\n Raises:\n ServiceStartTimeoutError: If the service takes too long to start.\n ServiceStartPortError: If the service writes an invalid port file or unable to read it.\n ServiceStartProcessError: If the service process exits unexpectedly.", "matching_document_document_id": "7b214499-9701-4d4e-95ff-34635c3b816e", "query_weight": 1.207741935483871 }, { "query_id": "e3606ac0-ff96-4b0b-a258-8024f823f0eb", "query": "How to effectively increase the timeout for wandb service?", "matching_document": "if not run:\n with self._jobs_lock:\n job_tracker.failed_to_start = True\n return\n with self._jobs_lock:\n job_tracker.run = run\n start_time = time.time()\n stopped_time: Optional[float] = None\n while self._jobs_event.is_set():\n # If run has failed to start before timeout, kill it\n state = (await run.get_status()).state\n if state == \"starting\" and RUN_START_TIMEOUT > 0:\n if time.time() - start_time > RUN_START_TIMEOUT:\n await run.cancel()\n raise LaunchError(\n f\"Run failed to start within {RUN_START_TIMEOUT} seconds. \"\n \"If you want to increase this timeout, set WANDB_LAUNCH_START_TIMEOUT \"\n \"to a larger value.\"\n )\n if await self._check_run_finished(job_tracker, launch_spec):", "matching_document_document_id": "1e0e36b8-722c-4430-b1d8-ad9fedc7e993", "query_weight": 1.207741935483871 }, { "query_id": "5416373a-bd17-41d4-a6f8-dd05feca6976", "query": "configuring wandb Reports to hide personal or sensitive data", "matching_document": "## Project Page\n### Project Defaults\n\nYou can change your project default settings *manually* in your User Settings at `/settings`. \n\n* **Default location to create new projects**: This is set to your own personal entity by default. By clicking on the dropdown, you can switch between your personal entity and the teams you're part of.\n* **Default project privacy in your personal account**: This is set to 'Private' by default. In other words, your projects will be private and can only be accessed by you.\n* **Enable code saving in your personal account**: This is turned off by default. You can turn this on to save the main script or notebook to W&B. \n\n:::note\nThese settings can also be specified by passing arguments to\n`wandb.init`.\n:::", "matching_document_document_id": "53941649-aa89-4884-8320-1063d1065860", "query_weight": 0.6967741935483871 }, { "query_id": "a903a709-491d-4534-aa8c-6eadf32b68e5", "query": "wandb report customization", "matching_document": "## Edit a report\n#### Add code blocks\nAdd code blocks to your report interactively with the App UI or with the W&B SDK. \n\nEnter a forward slash (`/`) in the report to display a dropdown menu. From the dropdown choose **Code**. \n\nSelect the name of the programming language on the right hand of the code block. This will expand a dropdown. From the dropdown, select your programming language syntax. You can choose from Javascript, Python, CSS, JSON, HTML, Markdown, and YAML. \n\nUse the `wr.CodeBlock` Class to create a code block programmatically. Provide the name of the language and the code you want to display for the language and code parameters, respectively. \n\nFor example the proceeding example demonstrates a list in YAML file: \n\n```\nimport wandb\nimport wandb.apis.reports as wr\n\nreport = wr.Report(project=\"report-editing\")\n\nreport.blocks = [\nwr.CodeBlock(\ncode=[\"this:\", \"- is\", \"- a\", \"cool:\", \"- yaml\", \"- file\"], language=\"yaml\"\n)\n]\n\nreport.save()\n\n``` \n\nThis will render a code block similar to:", "matching_document_document_id": "a39a3658-1ef3-4079-97f4-ace6ef9810b0", "query_weight": 0.6967741935483871 }, { "query_id": "e837f9fc-d80f-4c29-a521-ccf17de79474", "query": "privacy features in wandb for report generation", "matching_document": "## Collaborate on reports\n\nCollaborate and Share W&B Reports \n\nOnce you have saved a report, you can select the **Share** button to collaborate. A draft copy of the report is created when you select the **Edit** button. Draft reports auto-save. Select **Save to report** to publish your changes to the shared report. \n\nA warning notification will appear if an edit conflict occurs. This can occur if you and another collaborator edit the same report at the same time. The warning notification will guide you to resolve potential edit conflicts. \n\n### Comment on reports \n\nClick the comment button on a panel in a report to add a comment directly to that panel. \n\n### Who can edit and share reports? \n\nReports that are created within an individual's private project is only visible to that user. The user can share their project to a team or to the public.", "matching_document_document_id": "5bab537b-05eb-4ac3-9db7-27a98adcbed0", "query_weight": 0.6967741935483871 }, { "query_id": "b388f452-74de-4815-86fa-e93d37e4a75c", "query": "Are there any UI customization options for Trace Timeline in W&B Prompts?", "matching_document": "## Prompts for LLMs\n### Products\n#### Traces\nUse Traces for LLM chaining, plug-in or pipelining use cases. You can use your own LLM chaining implementation or use a W&B integration provided by LLM libraries such as LangChain. \n\nTraces consists of three main components: \n\n* Trace table: Overview of the inputs and outputs of a chain.\n* Trace timeline: Displays the execution flow of the chain and is color-coded according to component types.\n* Model architecture: View details about the structure of the chain and the parameters used to initialize each component of the chain. \n\n**Trace Table** \n\nThe Trace Table provides an overview of the inputs and outputs of a chain. The trace table also provides information about the composition of a trace event in the chain, whether or not the chain ran successfully, and any error messages returned when running the chain. \n\nClick on a row number on the left hand side of the Table to view the\u00a0Trace Timeline\u00a0for that instance of the chain. \n\n**Trace Timeline**", "matching_document_document_id": "c75507ef-d0dd-4a8f-9fe7-ba7917b5c973", "query_weight": 0.6967741935483871 }, { "query_id": "0c66da73-ce81-4cfb-93e7-65a3f25fa57b", "query": "grouped runs metrics", "matching_document": "## GCP with Weights & Biases\n### Advanced Features: Resuming and Grouping Runs\n#### Grouping Runs\nYou might want to group many different runs with respect to one or more configuration variables. This helps to draw comparisons between many different runs within a project or even distributed training. I am going to show a simple example of grouping runs together. \n\nGo to your project page (the URL should be like [https://app.wandb.ai/sayakpaul/arxiv-project-complex-models](https://app.wandb.ai/sayakpaul/arxiv-project-complex-models)) and press \u201cALT + Space\u201d. It should look like - \n\nNow, click on Group and you will see a list of the available configuration variables - \n\nNow, I wish to group the runs with respect to the learning rate which is present as lr. I will select it accordingly and I am done - \n\nYou can go beyond just one field and select any field you may find necessary to group together the runs for your purpose - \n\nNote: Be sure to turn off your notebook instance after you are done with your work -", "matching_document_document_id": "db882be2-e0b7-49da-95b3-c958fc34642b", "query_weight": 1.904516129032258 }, { "query_id": "eeb9c173-d6c0-4ab6-b408-8b5b927a9431", "query": "Troubleshooting Weights & Biases confusion matrix chart visualization", "matching_document": "## Step 2: Create a custom chart for the confusion matrix\nW&B custom charts are written in [Vega](https://vega.github.io/vega/), a powerful and flexible visualization language. You can find many examples and walkthroughs online, and it can help to start with an existing preset that is most similar to your desired custom visualization. You can iterate from small changes in our [IDE](https://wandb.ai/wandb/posts/reports/The-W-B-Machine-Learning-Visualization-IDE--VmlldzoyOTQxNDY5/edit), which renders the plot as you change its definition. \n\nHere is the full Vega spec for this multi-class confusion matrix: \n\n* From your project workspace or report, click on \"Add a visualization\" and select \"Custom chart\"\n* Pick any existing preset and replace its definition with the Vega spec below\n* Click \"Save As\" to give this preset a name for easier reference (I recommend \"confusion_matrix\" :)", "matching_document_document_id": "0c28a653-58b5-42dc-9742-7cb178bdee9b", "query_weight": 1.904516129032258 }, { "query_id": "02e389be-2ab4-48c2-85a3-c77244b8b9f7", "query": "How to troubleshoot a runs table rendering issue in Weights & Biases?", "matching_document": "## Runs Table\n### Filter and sort runs by minimum and maximum values\n1. Hover your mouse over the column with the metric you want to sort with.\n2. Select the kebob menu (three vertical lines).\n3. From the dropdown, select either **Show min** or **Show max**.\n4. From the same dropdown, select **Sort by asc** or **Sort by desc** to sort in ascending or descending order, respectively. \n\n#### What to do in case regex fails? \n\nIf regex doesn't provide you the desired results, you can make use of tags to filter out the runs in Runs Table. Tags can be added either on run creation or after they're finished. Once the tags are added to a run, you can add a tag filter as shown in the gif below.", "matching_document_document_id": "66565b01-fd04-47c4-a93a-074ae57d79ea", "query_weight": 1.904516129032258 }, { "query_id": "189377a3-c12d-48d9-842c-5bd3d8a7fe3d", "query": "Python training script", "matching_document": "## Quickstart\n### Putting it all together\n\nPutting it all together, your training script might look similar to the following code example. The highlighted code shows W&B-specific code.\nNote that we added code that mimics machine learning training. \n\n```\n# train.py\nimport wandb\nimport random # for demo script\n\n# highlight-next-line\nwandb.login()\n\nepochs = 10\nlr = 0.01\n\n# highlight-start\nrun = wandb.init(\n# Set the project where this run will be logged\nproject=\"my-awesome-project\",\n# Track hyperparameters and run metadata\nconfig={\n\"learning\\_rate\": lr,\n\"epochs\": epochs,\n},\n)\n# highlight-end\n\noffset = random.random() / 5\nprint(f\"lr: {lr}\")\n\n# simulating a training run\nfor epoch in range(2, epochs):\nacc = 1 - 2\\*\\*-epoch - random.random() / epoch - offset\nloss = 2\\*\\*-epoch + random.random() / epoch + offset\nprint(f\"epoch={epoch}, accuracy={acc}, loss={loss}\")\n# highlight-next-line\nwandb.log({\"accuracy\": acc, \"loss\": loss})\n\n# run.log\\_code()\n\n```", "matching_document_document_id": "9511a309-b09e-4f79-b9c7-943cb38a542b", "query_weight": 1.3006451612903225 }, { "query_id": "fc317b21-0851-4203-86a6-c1a1e6587152", "query": "wandb TensorFlow integration issue", "matching_document": "## Simple Integration With Any Framework\n### \ud83c\udf0a TensorFlow\nThe simplest way to log metrics in [TensorFlow](https://wandb.ai/fully-connected/blog/tf-lite) is by logging tf.summary with our TensorFlow logger: \n\n```\nimport wandb\n\n# 1. Start a W&B run\nwandb.init(project='gpt3')\n\n# 2. Save model inputs and hyperparameters\nconfig = wandb.config\nconfig.learning_rate = 0.01\n\n# Model training here\n\n# 3. Log metrics over time to visualize performance\nwith tf.Session() as sess:\n# ...\nwandb.tensorflow.log(tf.summary.merge_all())\n``` \n\n*\n*", "matching_document_document_id": "71a8fb62-c21f-4770-990d-c3f600e4fb9a", "query_weight": 1.3006451612903225 }, { "query_id": "70c12510-0686-475e-a076-76c4ddd523cb", "query": "Integrating Weights & Biases plots without reports", "matching_document": "' \n\n# Creating Custom Line Plots With Weights & Biases \n\nDescription: This article provides usage and examples for wandb.plot.line(), explaining how to log a custom line plot natively in just a few lines. \n\nBody: \n\n# Method: wandb.plot.line() \n\nLog a custom line plot\u2014a list of connected/ordered points (x, y) on a pair of arbitrary axes x and y\u2014natively in a few lines: \n\n```\ndata = [[x, y] for (x, y) in zip(x_values, y_values)]\ntable = wandb.Table(data=data, columns = [\"x\", \"y\"])\nwandb.log({\"my_custom_plot_id\" : wandb.plot.line(table,\n\"x\", \"y\", title=\"Custom Y vs X Line Plot\")})\n``` \n\nYou can use this to log curves on any two dimensions. Note that if you're plotting two lists of values against each other, the number of values in the lists must match exactly (i.e. each point must have an x and a y). \n\n# Basic Usage Example \n\n# Customized Usage \n\nThere are many ways to customize the line plot using the [Vega visualization grammar](https://vega.github.io/). \n\nHere are some simple ones:", "matching_document_document_id": "8727d4a1-a537-4f65-a154-c1cb409f4c3b", "query_weight": 0.8825806451612903 }, { "query_id": "5a10df62-fe1f-4428-ad66-b45ead729f48", "query": "webpage integration", "matching_document": "def main():\n st.title(\"Streamlit w/ WANDB\")\n menu = [\"Embed WANDB\", \"Use WANDB Logging\"]\n menu_choice = st.sidebar.selectbox('Menu', menu)\n\n entity = os.environ.get(\"WANDB_ENTITY\", \"demo-user\")\n height = 720\n\n projects = get_projects(entity, height=720)\n if menu_choice == 'Embed WANDB':\n st.subheader(\"WANDB IFrame test\")\n\n # Get list of projects for provided entity whose API key matches\n # Show all those list of projects as selectable options in the sidebar while also grabbing the Iframe link\n # Then display the iframe of the selected project\n selected_project = st.sidebar.selectbox(\n \"Project Name\", list(projects.keys()))\n selected_project_iframe = projects[selected_project]\n st.text_area(\"Using the WANDB API we can directly query project and run page links which we can use to embed Iframes\")\n st.subheader(\"PROJECT DETAILS:\")\n components.html(selected_project_iframe, height=height)", "matching_document_document_id": "fa3dc63a-7a6e-4596-b795-b9bc3509fa89", "query_weight": 0.8825806451612903 }, { "query_id": "b98fd924-ebeb-469e-bcf4-2181618e3404", "query": "Tutorial on extending wandb.Table with new columns", "matching_document": "def add_column(self, name, data, optional=False):\n \"\"\"Adds a column of data to the table.\n\n Arguments:\n name: (str) - the unique name of the column\n data: (list | np.array) - a column of homogenous data\n optional: (bool) - if null-like values are permitted\n \"\"\"\n assert isinstance(name, str) and name not in self.columns\n is_np = util.is_numpy_array(data)\n assert isinstance(data, list) or is_np\n assert isinstance(optional, bool)\n is_first_col = len(self.columns) == 0\n assert is_first_col or len(data) == len(\n self.data\n ), f\"Expected length {len(self.data)}, found {len(data)}\"\n\n # Add the new data\n for ndx in range(max(len(data), len(self.data))):\n if is_first_col:\n self.data.append([])\n if is_np:\n self.data[ndx].append(data[ndx])\n else:\n self.data[ndx].append(data[ndx])\n # add the column\n self.columns.append(name)", "matching_document_document_id": "ccb0ddd3-392d-4d5b-94d0-69f4fe4d367d", "query_weight": 0.8361290322580645 }, { "query_id": "26afc52a-b7d1-4c6c-827e-40242cb803ba", "query": "Performing arithmetic operations on wandb.Table columns", "matching_document": "def add_computed_columns(self, fn):\n \"\"\"Adds one or more computed columns based on existing data.\n\n Args:\n fn: A function which accepts one or two parameters, ndx (int) and row (dict),\n which is expected to return a dict representing new columns for that row, keyed\n by the new column names.\n\n `ndx` is an integer representing the index of the row. Only included if `include_ndx`\n is set to `True`.\n\n `row` is a dictionary keyed by existing columns\n \"\"\"\n new_columns = {}\n for ndx, row in self.iterrows():\n row_dict = {self.columns[i]: row[i] for i in range(len(self.columns))}\n new_row_dict = fn(ndx, row_dict)\n assert isinstance(new_row_dict, dict)\n for key in new_row_dict:\n new_columns[key] = new_columns.get(key, [])\n new_columns[key].append(new_row_dict[key])\n for new_col_name in new_columns:\n self.add_column(new_col_name, new_columns[new_col_name])", "matching_document_document_id": "89e3467a-631a-4f55-a831-0507e6a4a717", "query_weight": 0.8361290322580645 }, { "query_id": "c3969352-9a61-4089-a5bb-4d2d50e50307", "query": "column operations wandb", "matching_document": "def add_computed_columns(self, fn):\n \"\"\"Adds one or more computed columns based on existing data.\n\n Args:\n fn: A function which accepts one or two parameters, ndx (int) and row (dict),\n which is expected to return a dict representing new columns for that row, keyed\n by the new column names.\n\n `ndx` is an integer representing the index of the row. Only included if `include_ndx`\n is set to `True`.\n\n `row` is a dictionary keyed by existing columns\n \"\"\"\n new_columns = {}\n for ndx, row in self.iterrows():\n row_dict = {self.columns[i]: row[i] for i in range(len(self.columns))}\n new_row_dict = fn(ndx, row_dict)\n assert isinstance(new_row_dict, dict)\n for key in new_row_dict:\n new_columns[key] = new_columns.get(key, [])\n new_columns[key].append(new_row_dict[key])\n for new_col_name in new_columns:\n self.add_column(new_col_name, new_columns[new_col_name])", "matching_document_document_id": "89e3467a-631a-4f55-a831-0507e6a4a717", "query_weight": 0.8361290322580645 }, { "query_id": "197a89ac-f670-4545-aa35-d903e70f2f41", "query": "wandb.Table data manipulation", "matching_document": "def add_computed_columns(self, fn):\n \"\"\"Adds one or more computed columns based on existing data.\n\n Args:\n fn: A function which accepts one or two parameters, ndx (int) and row (dict),\n which is expected to return a dict representing new columns for that row, keyed\n by the new column names.\n\n `ndx` is an integer representing the index of the row. Only included if `include_ndx`\n is set to `True`.\n\n `row` is a dictionary keyed by existing columns\n \"\"\"\n new_columns = {}\n for ndx, row in self.iterrows():\n row_dict = {self.columns[i]: row[i] for i in range(len(self.columns))}\n new_row_dict = fn(ndx, row_dict)\n assert isinstance(new_row_dict, dict)\n for key in new_row_dict:\n new_columns[key] = new_columns.get(key, [])\n new_columns[key].append(new_row_dict[key])\n for new_col_name in new_columns:\n self.add_column(new_col_name, new_columns[new_col_name])", "matching_document_document_id": "89e3467a-631a-4f55-a831-0507e6a4a717", "query_weight": 0.8361290322580645 }, { "query_id": "bd6ee380-e615-4859-8b54-caf4c945980e", "query": "wandb.Table add new column based on existing columns", "matching_document": "## Table\n```\nimport pandas as pd\nimport wandb\n\ndata = {\"users\": [\"geoff\", \"juergen\", \"ada\"], \"feature\\_01\": [1, 117, 42]}\ndf = pd.DataFrame(data)\n\ntbl = wandb.Table(data=df)\nassert all(tbl.get\\_column(\"users\") == df[\"users\"])\nassert all(tbl.get\\_column(\"feature\\_01\") == df[\"feature\\_01\"])\n\n``` \n\nAdditionally, users can add data to Tables incrementally by using the\n`add_data`, `add_column`, and `add_computed_column` functions for\nadding rows, columns, and columns computed from data in other columns, respectively: \n\n```\nimport wandb\n\ntbl = wandb.Table(columns=[\"user\"])\n\nusers = [\"geoff\", \"juergen\", \"ada\"]\n\n[tbl.add\\_data(user) for user in users]\nassert tbl.get\\_column(\"user\") == users\n\ndef get\\_user\\_name\\_length(index, row):\nreturn {\"feature\\_01\": len(row[\"user\"])}\n\ntbl.add\\_computed\\_columns(get\\_user\\_name\\_length)\nassert tbl.get\\_column(\"feature\\_01\") == [5, 7, 3]\n\n``` \n\nTables can be logged directly to runs using `run.log({\"my_table\": table})`\nor added to artifacts using `artifact.add(table, \"my_table\")`:", "matching_document_document_id": "b915a272-8ca7-4a47-bb32-617e7d36e7ba", "query_weight": 0.8361290322580645 }, { "query_id": "62ae4c2e-2147-4fa1-8a75-56624b06ae6e", "query": "Runs view UI customization", "matching_document": "## Runs\n### View all Runs in a Project\n* **Kebob menu**: Use this kebob menu to rename a Run, delete a Run, or stop an active Run.\n* **Visibility icon**: Select the eye icon to hide specific run.\n* **Color**: change the run color to another one of our presets or a custom color.\n* **Search**: search runs by name. This also filters visible runs in the plots.\n* **Filter**: use the sidebar filter to narrow down the set of runs visible.\n* **Group**: select a config column to dynamically group your runs, for example by architecture. Grouping makes plots show up with a line along the mean value, and a shaded region for the variance of points on the graph.\n* **Sort**: pick a value to sort your runs by, for example runs with the lowest loss or highest accuracy. Sorting will affect which runs show up on the graphs.\n* **Expand button**: expand the sidebar into the full table\n* **Run count**: the number in parentheses at the top is the total number of runs in the project. The number (N visualized) is the number of runs that have the eye turned on and are available to be visualized in each plot. In the example below, the graphs are only showing the first 10 of 183 runs. Edit a graph to increase the max number of runs visible.", "matching_document_document_id": "4e78ec8c-0716-419e-a4a6-2443e88f10ba", "query_weight": 0.6967741935483871 }, { "query_id": "fcfe7fbd-4899-40cd-b089-b4783cde639f", "query": "What steps are needed to customize the UI of the W&B Workspace for better visibility of runs?", "matching_document": "## Customized Usage\nHere are some simple ones: \n\n* change the appearance and opacity of the point markers: change `\"mark\": {\"type\": \"circle\",` to `\"mark\": {\"type\": \"point\", \"opacity\" : 0.3,`\n* rename the axis titles for clarity: add `\"title\" : \"Your Title\"` to the `x` and `y` fields under `encoding` \n\nHere is a pretty advanced one: set opacity conditionally based on the prefix of your run names, so you can display two different run sets in one-panel section (previously impossible in W&B). You will need to create two copies of the panel and the Vega spec and modify the test condition in one of the copies to make the inverse set of runs invisible (i.e. replace \"frog\" with \"bug\"). \n\n```\n\"opacity\" : {\n\"condition\" :{\n\"test\" : \"indexof(datum.name, 'frog') > -1\",\n\"value\" : 0}\n}\n}\n```", "matching_document_document_id": "285ab3e2-c463-43d7-ab93-3f1d7fa0037c", "query_weight": 0.6967741935483871 }, { "query_id": "7aa3fbe8-b174-4e30-887a-b2c9fc7cf44c", "query": "Best practices for using multiple Weights & Biases accounts", "matching_document": "## Organizations\n\nOrganizations allow you to manage membership across teams in your organization as well as subscription settings and billing information. \n\n:::note\nW&B recommends to have more than one admin in an organization. It is a best practice to ensure that admin operations can continue when the primary admin is not available.\n::: \n\n## Invite organization members \n\nInvite new members to your organizations.\n1. Ensure the team member already has a W&B Account.\n2. Navigate to .\n3. Select **Manage members**.\n4. A model will appear. Provide the username or email for the **Email or Username** field, select a team for them to join from the **Team** dropdown menu, and select a role type from the **Organizational Role** dropdown menu. \n\n1. Select the **Add** button. \n\n:::info\n\\* If you have an Enterprise account, please contact your Account Executive to invite new members to your team.\n::: \n\n## Change the account settings for an organization", "matching_document_document_id": "6dff1d2a-1f5b-4bbe-b61a-53e838878a6f", "query_weight": 0.6967741935483871 }, { "query_id": "0a5cf925-2080-4a07-9fc4-fe068d389666", "query": "Server-side authentication methods for Weights & Biases", "matching_document": "## Single Sign-On (SSO) setup\n### Configure SSO on the W&B App\nOnce you have everything configured you can provide the Issuer, Client ID, and Auth method to `wandb/local` on the W&B App or set environment variables. The following procedure walks you through the steps to configure SSO with the W&B App UI: \n\n1. Sign in to your Weights and Biases server\n2. Navigate to the W&B App. \n\n1. From the dropdown, select **System Settings**: \n\n1. Enter your Issuer, Client ID, and Authentication Method.\n2. Select **Update settings**. \n\n:::info\nIf you're unable to log in to your instance after configuring SSO, you can restart the instance with the `LOCAL_RESTORE=true` environment variable set. This will output a temporary password to the containers logs and disable SSO. Once you've resolved any issues with SSO, you must remove that environment variable to enable SSO again.\n:::", "matching_document_document_id": "ed049abf-2315-4b12-8ae4-007ea332c65b", "query_weight": 0.6967741935483871 }, { "query_id": "a2e2a9b0-129c-4a4f-acf4-cdad49b172fd", "query": "multi-user collaboration on W&B tables", "matching_document": "--- \n\n## description: Collaborate and share W&B Reports with peers, co-workers, and your team. \n\n# Collaborate on reports \n\nCollaborate and Share W&B Reports \n\nOnce you have saved a report, you can select the **Share** button to collaborate. A draft copy of the report is created when you select the **Edit** button. Draft reports auto-save. Select **Save to report** to publish your changes to the shared report. \n\nA warning notification will appear if an edit conflict occurs. This can occur if you and another collaborator edit the same report at the same time. The warning notification will guide you to resolve potential edit conflicts. \n\n### Comment on reports \n\nClick the comment button on a panel in a report to add a comment directly to that panel. \n\n### Who can edit and share reports? \n\nReports that are created within an individual's private project is only visible to that user. The user can share their project to a team or to the public.", "matching_document_document_id": "7fe941de-f8fe-4acc-b460-004c8b6a298e", "query_weight": 0.6503225806451612 }, { "query_id": "203f64d4-0c45-4da2-8a4f-eb1c867db3ae", "query": "Troubleshooting wandb Sweep errors when using multiple accounts", "matching_document": "## Troubleshoot Sweeps\n\nTroubleshoot W&B Sweeps \n\nTroubleshoot common error messages with the guidance suggested. \n\n### `CommError, Run does not exist` and `ERROR Error uploading` \n\nYour W&B Run ID might be defined if these two error messages are both returned. As an example, you might have a similar code snippet defined somewhere in your Jupyter Notebooks or Python script: \n\n```\nwandb.init(id=\"some-string\")\n\n``` \n\nYou can not set a Run ID for W&B Sweeps because W&B automatically generates random, unique IDs for Runs created by W&B Sweeps. \n\nW&B Run IDs need to be unique within a project. \n\nWe recommend you pass a name to the name parameter when you initialized W&B, if you want to set a custom name that will appear on tables and graphs. For example: \n\n```\nwandb.init(name=\"a helpful readable run name\")\n\n``` \n\n### `Cuda out of memory` \n\n### `anaconda 400 error` \n\nThe following error usually occurs when you do not log the metric that you are optimizing:", "matching_document_document_id": "13152c97-59c9-456d-b955-55a590204565", "query_weight": 0.9290322580645161 }, { "query_id": "c82f7292-7c3e-40e8-9244-d0f08dc0c7c5", "query": "configure wandb for multiple users", "matching_document": "--- \n\n## displayed\\_sidebar: default \n\n# Manage users \n\nManage W&B users in your organization or team. \n\nW&B strongly recommends and encourages that users authenticate to an enterprise W&B Server instance using Single Sign-On (SSO). To learn more about how to setup SSO with W&B Server, refer to the SSO Configuration documentation. \n\n:::note\n`W&B Server` refers to both **Dedicated Cloud** or **Self-managed** hosting options.\n::: \n\n:::note\n`Instance` or `organization` terms are used interchangeably within the context of W&B Server. \n\nW&B is actively developing support for multiple organizations in an enterprise instance of W&B Server. If you're interested in utilizing that capability, reach out to your W&B team.\n::: \n\n## Instance Admins \n\nThe first user to sign up after the W&B Server instance is initially deployed, is automatically assigned the instance `admin` role. The admin can then add additional users to the organization and create teams. \n\n## Manage your organization \n\n## Manage a team", "matching_document_document_id": "7134b9c4-847d-4000-b82a-e337890f7c3d", "query_weight": 0.9290322580645161 }, { "query_id": "b98a920d-a7ab-4b13-a753-ecb3bd22c900", "query": "Best practices for using wandb Sweeps across different machines and accounts", "matching_document": "## Parallelize agents\n#### Parallelize on a multi-GPU machine\n1. Open more than one terminal window on your local machine.\n2. Specify the GPU instance to use with `CUDA_VISIBLE_DEVICES` when you start a W&B Sweep job (`wandb agent`). Assign `CUDA_VISIBLE_DEVICES` an integer value corresponding to the GPU instance to use. \n\nFor example, suppose you have two NVIDIA GPUs on your local machine. Open a terminal window and set `CUDA_VISIBLE_DEVICES` to `0` (`CUDA_VISIBLE_DEVICES=0`). Replace `sweep_ID` in the proceeding example with the W&B Sweep ID that is returned when you initialized a W&B Sweep: \n\nTerminal 1 \n\n```\nCUDA\\_VISIBLE\\_DEVICES=0 wandb agent sweep_ID\n\n``` \n\nOpen a second terminal window. Set `CUDA_VISIBLE_DEVICES` to `1` (`CUDA_VISIBLE_DEVICES=1`). Paste the same W&B Sweep ID for the `sweep_ID` mentioned in the proceeding code snippet: \n\nTerminal 2 \n\n```\nCUDA\\_VISIBLE\\_DEVICES=1 wandb agent sweep_ID\n\n```", "matching_document_document_id": "03228a82-a0d9-405b-ba85-e645187827e1", "query_weight": 0.9290322580645161 }, { "query_id": "47e51dd3-dafe-43a5-bb32-338d2961e0bb", "query": "How to share a wandb Sweep ID across different accounts?", "matching_document": "## Set Up Your Own spaCy Project To Use W&B Sweeps\n### Call the Training Code With Sweeps \ud83e\uddf9\n#### Full Sweeps Config\n```\nmethod: bayes\nmetric:\ngoal: maximize\nname: cats_macro_auc\ncommand:\n- ${env}\n- ${interpreter}\n- scripts/sweeps_using_config.py\n- ./configs/default_config.cfg\n- ./training\nparameters:\ncomponents.textcat.model.conv_depth:\nvalues:\n- 2\n- 3\n- 4\ncomponents.textcat.model.ngram_size:\nvalues:\n- 1\n- 2\n- 3\ntraining.dropout:\ndistribution: uniform\nmax: 0.5\nmin: 0.05\ntraining.optimizer.learn_rate:\ndistribution: uniform\nmax: 0.01\nmin: 0.001\n``` \n\n> \n\nNow that we have defined our sweeps configuration file, we can call it using the wandb CLI like this: \n\n```\nwandb sweep my_sweep.yml\n``` \n\nwhich will output a sweep-id which you should pass to wandb agent: \n\n```\nwandb agent sweep-id\n``` \n\nYou can call this on each machine or within each process that you'd like to contribute to the sweep. \n\nAnd that's it. Now you can get back to the other, harder-to-automate parts of your job.", "matching_document_document_id": "38a0c305-7648-4d04-89ff-af41b8beeb14", "query_weight": 0.9290322580645161 }, { "query_id": "2c98edec-a3fb-4553-affa-0e3b65e6df12", "query": "handling API errors in Weights & Biases", "matching_document": "class AuthenticationError(CommError):\n \"\"\"Raised when authentication fails.\"\"\"\n\nclass CommError(Error):\n \"\"\"Error communicating with W&B servers.\"\"\"\n\n def __init__(self, msg, exc=None) -> None:\n self.exc = exc\n self.message = msg\n super().__init__(self.message)\n\nclass Error(Exception):\n \"\"\"Base W&B Error.\"\"\"\n\n def __init__(self, message, context: Optional[dict] = None) -> None:\n super().__init__(message)\n self.message = message\n # sentry context capture\n if context:\n self.context = context\n\nclass UnsupportedError(UsageError):\n \"\"\"Raised when trying to use a feature that is not supported.\"\"\"\n\nclass UsageError(Error):\n \"\"\"Raised when an invalid usage of the SDK API is detected.\"\"\"", "matching_document_document_id": "a6339965-82ea-4100-97f7-292cfe070961", "query_weight": 0.7896774193548387 }, { "query_id": "b6a7c32b-1f27-435a-8148-da991bb358c2", "query": "Optimizing Weights & Biases API calls for better performance", "matching_document": "## Limits & Performance\n### Best Practices for Fast Pages\n#### Python Script Performance\n\nThere are few common ways the performance of your python script can be reduced: \n\n1. The size of your data is too large. Large data sizes could introduce a >1ms overhead to the training loop.\n2. The speed of your network and the how the W&B backend is configured\n3. Calling `wandb.log` more than a few times per second. This is due to a small latency added to the training loop every time `wandb.log` is called. \n\n:::info\nIs frequent logging slowing your training runs down? Check out this Colab for methods to get better performance by changing your logging strategy.\n::: \n\nWe do not assert any limits beyond rate limiting. Our Python client will automatically do an exponential backoff and retry requests that exceed limits, so this should be transparent to you. It will say \u201cNetwork failure\u201d on the command line. For unpaid accounts, we may reach out in extreme cases where usage exceeds reasonable thresholds.", "matching_document_document_id": "b0998919-90bf-44b2-9461-acccc2f7f4e2", "query_weight": 0.7896774193548387 }, { "query_id": "ff2ee0db-b72c-4195-9149-d795f47f86f1", "query": "Best practices for API error management in Weights & Biases", "matching_document": "## Troubleshoot Sweeps\n\nTroubleshoot W&B Sweeps \n\nTroubleshoot common error messages with the guidance suggested. \n\n### `CommError, Run does not exist` and `ERROR Error uploading` \n\nYour W&B Run ID might be defined if these two error messages are both returned. As an example, you might have a similar code snippet defined somewhere in your Jupyter Notebooks or Python script: \n\n```\nwandb.init(id=\"some-string\")\n\n``` \n\nYou can not set a Run ID for W&B Sweeps because W&B automatically generates random, unique IDs for Runs created by W&B Sweeps. \n\nW&B Run IDs need to be unique within a project. \n\nWe recommend you pass a name to the name parameter when you initialized W&B, if you want to set a custom name that will appear on tables and graphs. For example: \n\n```\nwandb.init(name=\"a helpful readable run name\")\n\n``` \n\n### `Cuda out of memory` \n\n### `anaconda 400 error` \n\nThe following error usually occurs when you do not log the metric that you are optimizing:", "matching_document_document_id": "13152c97-59c9-456d-b955-55a590204565", "query_weight": 0.7896774193548387 }, { "query_id": "66c4f5f3-bbf7-4290-95ca-fa9a3b0898b3", "query": "Best practices for error handling in Weights & Biases API requests", "matching_document": "## Troubleshooting\n#### How do I deal with network issues?\n\nIf you're seeing SSL or network errors:`wandb: Network error (ConnectionError), entering retry loop`. You can try a couple of different approaches to solving this issue: \n\n1. Upgrade your SSL certificate. If you're running the script on an Ubuntu server, run `update-ca-certificates` We can't sync training logs without a valid SSL certificate because it's a security vulnerability.\n2. If your network is flaky, run training in offline mode and sync the files to us from a machine that has Internet access.\n3. Try running W&B Private Hosting, which operates on your machine and doesn't sync files to our cloud servers. \n\n`SSL CERTIFICATE_VERIFY_FAILED`: this error could be due to your company's firewall. You can set up local CAs and then use: \n\n`export REQUESTS_CA_BUNDLE=/etc/ssl/certs/ca-certificates.crt`", "matching_document_document_id": "eba9b6dc-4582-4539-bf24-39b10a2e6960", "query_weight": 0.7896774193548387 }, { "query_id": "6184c621-d589-4cc2-8fc4-54c653066789", "query": "Steps to troubleshoot variables not appearing in W&B dashboard during a sweep.", "matching_document": "## Troubleshoot Sweeps\n\nTroubleshoot W&B Sweeps \n\nTroubleshoot common error messages with the guidance suggested. \n\n### `CommError, Run does not exist` and `ERROR Error uploading` \n\nYour W&B Run ID might be defined if these two error messages are both returned. As an example, you might have a similar code snippet defined somewhere in your Jupyter Notebooks or Python script: \n\n```\nwandb.init(id=\"some-string\")\n\n``` \n\nYou can not set a Run ID for W&B Sweeps because W&B automatically generates random, unique IDs for Runs created by W&B Sweeps. \n\nW&B Run IDs need to be unique within a project. \n\nWe recommend you pass a name to the name parameter when you initialized W&B, if you want to set a custom name that will appear on tables and graphs. For example: \n\n```\nwandb.init(name=\"a helpful readable run name\")\n\n``` \n\n### `Cuda out of memory` \n\n### `anaconda 400 error` \n\nThe following error usually occurs when you do not log the metric that you are optimizing:", "matching_document_document_id": "13152c97-59c9-456d-b955-55a590204565", "query_weight": 0.7896774193548387 }, { "query_id": "3a5d4acb-4a27-47ac-bf92-3a9a57ae9b17", "query": "How to ensure variables are correctly logged in W&B sweeps?", "matching_document": "## instead of defining hard values\nCopy the sweep ID and replace `sweepID` in the proceeding code snippet to start the sweep job with the `wandb agent` command: \n\n```\nwandb agent --count $NUM your-entity/sweep-demo-cli/sweepID\n\n``` \n\nFor more information on how to start sweep jobs, see Start sweep jobs. \n\n## Consideration when logging metrics \n\nEnsure to log the metric you specify in your sweep configuration explicitly to W&B. Do not log metrics for your sweep inside of a sub-directory. \n\nFor example, consider the proceeding psuedocode. A user wants to log the validation loss (`\"val_loss\": loss`). First they pass the values into a dictionary (line 16). However, the dictionary passed to `wandb.log` does not explicitly access the key-value pair in the dictionary:", "matching_document_document_id": "529f4145-2639-44b3-9e84-285f9b520c49", "query_weight": 0.7896774193548387 }, { "query_id": "54195908-e235-4b08-bc42-d987a7acbf9a", "query": "table merge techniques", "matching_document": "## Visualize & analyze tables\n### How to view two tables\n#### Merged view\n* **choose the join key**: use the dropdown at the top left to set the column to use as the join key for the two tables. Typically this will be the unique identifier of each row, such as the file name of a specific example in your dataset or an incrementing index on your generated samples. Note that it's currently possible to select *any* column, which may yield illegible tables and slow queries.\n* **concatenate instead of join**: select \"concatenating all tables\" in this dropdown to *union all the rows* from both tables into one larger Table instead of joining across their columns\n* **reference each Table explicitly**: use 0, 1, and \\* in the filter expression to explicitly specify a column in one or both table instances\n* **visualize detailed numerical differences as histograms**: compare the values in any cell at a glance", "matching_document_document_id": "1c62433f-54a0-421f-9173-57fad0943836", "query_weight": 1.2541935483870967 }, { "query_id": "70369238-5c34-443e-af98-df311d63f99e", "query": "joining data frames in pandas Python", "matching_document": "## Data preparation\n### Pre-process data for binary classification\nwe then open and convert the `json` file to a pandas `DataFrame` \n\n```\nannotations = pd.DataFrame.from\\_dict(data['annotations'])\nimages = pd.DataFrame.from\\_dict(data['images'])\n\n``` \n\n```\nannotations.head()\n\n``` \n\n```\nimages.head()\n\n``` \n\n```\ndf = annotations[['image\\_id', 'category\\_id']].groupby('image\\_id')['category\\_id'].apply(lambda x: list(set(x))).reset\\_index()\ndf.head()\n\n``` \n\n```\ndf['mold'] = df['category\\_id'].apply(lambda x: 4 in x)\ndf['mold'].value\\_counts()\n\n``` \n\n```\ndf = pd.merge(df, images[['id', 'file\\_name']], left\\_on='image\\_id', right\\_on='id')\n\n``` \n\n```\ndel df['id']\n\n``` \n\n```\ndf['fruit\\_id'] = df['file\\_name'].apply(lambda x: x.split('/')[1].split('\\_')[0])\n\n``` \n\n```\ndf.head()\n\n```", "matching_document_document_id": "bb893b78-6424-4e8b-b758-f8cd72e2affc", "query_weight": 1.2541935483870967 }, { "query_id": "1f75e688-eb70-489b-8583-cbd8d3ba0572", "query": "How to join two tables in Python using pandas", "matching_document": "## Visualize & analyze tables\n### How to view two tables\n#### Merged view\n* **choose the join key**: use the dropdown at the top left to set the column to use as the join key for the two tables. Typically this will be the unique identifier of each row, such as the file name of a specific example in your dataset or an incrementing index on your generated samples. Note that it's currently possible to select *any* column, which may yield illegible tables and slow queries.\n* **concatenate instead of join**: select \"concatenating all tables\" in this dropdown to *union all the rows* from both tables into one larger Table instead of joining across their columns\n* **reference each Table explicitly**: use 0, 1, and \\* in the filter expression to explicitly specify a column in one or both table instances\n* **visualize detailed numerical differences as histograms**: compare the values in any cell at a glance", "matching_document_document_id": "1c62433f-54a0-421f-9173-57fad0943836", "query_weight": 1.2541935483870967 }, { "query_id": "5c629483-4e6f-4c4d-80f3-c1d6ee8de3cb", "query": "Are there specific libraries or tools recommended for merging tables?", "matching_document": "## Weave\n### Basic operations\n#### Join\n\nIt is also possible to join tables directly in the query, where:\n\\* `project(\"luis_team_test\", \"weave_example_queries\").runs.summary[\"short_table_0\"].table.rows.concat` is the first table\n\\* `project(\"luis_team_test\", \"weave_example_queries\").runs.summary[\"short_table_1\"].table.rows.concat` is the second table\n\\* `(row) => row[\"Label\"]` are selectors for each table, determining which column to join on\n\\* `\"Table1\"` and `\"Table2\"` are the names of each table when joined\n\\* `true` and `false` are for left and right inner/outer join settings", "matching_document_document_id": "a1d148a0-a973-4820-ace9-8e3c0e14aaf3", "query_weight": 1.2541935483870967 }, { "query_id": "b85ff1c4-f236-4bb0-aba2-896fa8f7c586", "query": "Using regex to exclude data in search", "matching_document": "## Runs Table\n\nOn the project page, we show runs in a sidebar. Expand the sidebar to view a table of hyperparameters and summary metrics across runs. \n\n## Search run names \n\nWe support full regex search on run names in the table. When you type a query in the search box, that will filter down the visible runs in the graphs on the workspace as well as filtering the rows of the table. \n\n## Filter and sort runs by minimum and maximum values \n\nSort the runs table by the minimum or maximum value of a logged metric. This is particularly useful if you want to view the best (or worst) recorded value. \n\nThe following steps describe how to sort the run table by a specific metric based on the minimum or maximum recorded value: \n\n## Search End Time for runs \n\nWe provide a column named `End Time` that logs that last heartbeat from the client process. The field is hidden by default. \n\n## Resize the sidebar \n\n## Add sidebar columns \n\nOn the project page, we show runs in a sidebar. To show more columns:", "matching_document_document_id": "06f1e751-0ab0-4629-820a-e1bc5bbb114c", "query_weight": 0.8825806451612903 }, { "query_id": "11fd3f41-507e-40c5-9785-2f435a128d2d", "query": "Using Weights & Biases API to filter and retrieve specific runs", "matching_document": "## Api\n### Methods\n#### `runs`\n\nView source \n\n```\nruns(\npath: Optional[str] = None,\nfilters: Optional[Dict[str, Any]] = None,\norder: str = \"-created\\_at\",\nper\\_page: int = 50,\ninclude\\_sweeps: bool = (True)\n)\n\n``` \n\nReturn a set of runs from a project that match the filters provided. \n\nYou can filter by `config.*`, `summary_metrics.*`, `tags`, `state`, `entity`, `createdAt`, etc. \n\n#### Examples: \n\nFind runs in my\\_project where config.experiment\\_name has been set to \"foo\" \n\n```\napi.runs(path=\"my_entity/my_project\", filters={\"config.experiment_name\": \"foo\"})\n\n``` \n\nFind runs in my\\_project where config.experiment\\_name has been set to \"foo\" or \"bar\" \n\n```\napi.runs(\npath=\"my_entity/my_project\",\nfilters={\"$or\": [{\"config.experiment_name\": \"foo\"}, {\"config.experiment_name\": \"bar\"}]}\n)\n\n``` \n\nFind runs in my\\_project where config.experiment\\_name matches a regex (anchors are not supported) \n\n```\napi.runs(\npath=\"my_entity/my_project\",\nfilters={\"config.experiment_name\": {\"$regex\": \"b.*\"}}\n)\n\n```", "matching_document_document_id": "31bde459-9f43-4aa2-a114-158e8e91cb6f", "query_weight": 0.8825806451612903 }, { "query_id": "bab28976-3f32-44f4-8b88-ed03b2a28090", "query": "Using regex to filter runs in Weights & Biases", "matching_document": "## Runs Table\n\nOn the project page, we show runs in a sidebar. Expand the sidebar to view a table of hyperparameters and summary metrics across runs. \n\n## Search run names \n\nWe support full regex search on run names in the table. When you type a query in the search box, that will filter down the visible runs in the graphs on the workspace as well as filtering the rows of the table. \n\n## Filter and sort runs by minimum and maximum values \n\nSort the runs table by the minimum or maximum value of a logged metric. This is particularly useful if you want to view the best (or worst) recorded value. \n\nThe following steps describe how to sort the run table by a specific metric based on the minimum or maximum recorded value: \n\n## Search End Time for runs \n\nWe provide a column named `End Time` that logs that last heartbeat from the client process. The field is hidden by default. \n\n## Resize the sidebar \n\n## Add sidebar columns \n\nOn the project page, we show runs in a sidebar. To show more columns:", "matching_document_document_id": "06f1e751-0ab0-4629-820a-e1bc5bbb114c", "query_weight": 0.8825806451612903 }, { "query_id": "17086a78-1319-4a52-bef9-b460258da867", "query": "advanced search runs", "matching_document": "## Runs Table\n\nOn the project page, we show runs in a sidebar. Expand the sidebar to view a table of hyperparameters and summary metrics across runs. \n\n## Search run names \n\nWe support full regex search on run names in the table. When you type a query in the search box, that will filter down the visible runs in the graphs on the workspace as well as filtering the rows of the table. \n\n## Filter and sort runs by minimum and maximum values \n\nSort the runs table by the minimum or maximum value of a logged metric. This is particularly useful if you want to view the best (or worst) recorded value. \n\nThe following steps describe how to sort the run table by a specific metric based on the minimum or maximum recorded value: \n\n## Search End Time for runs \n\nWe provide a column named `End Time` that logs that last heartbeat from the client process. The field is hidden by default. \n\n## Resize the sidebar \n\n## Add sidebar columns \n\nOn the project page, we show runs in a sidebar. To show more columns:", "matching_document_document_id": "06f1e751-0ab0-4629-820a-e1bc5bbb114c", "query_weight": 0.8825806451612903 }, { "query_id": "e23d2b64-a70e-4621-b8be-9357c579e980", "query": "exclude runs with string", "matching_document": "## Runs Table\n\nOn the project page, we show runs in a sidebar. Expand the sidebar to view a table of hyperparameters and summary metrics across runs. \n\n## Search run names \n\nWe support full regex search on run names in the table. When you type a query in the search box, that will filter down the visible runs in the graphs on the workspace as well as filtering the rows of the table. \n\n## Filter and sort runs by minimum and maximum values \n\nSort the runs table by the minimum or maximum value of a logged metric. This is particularly useful if you want to view the best (or worst) recorded value. \n\nThe following steps describe how to sort the run table by a specific metric based on the minimum or maximum recorded value: \n\n## Search End Time for runs \n\nWe provide a column named `End Time` that logs that last heartbeat from the client process. The field is hidden by default. \n\n## Resize the sidebar \n\n## Add sidebar columns \n\nOn the project page, we show runs in a sidebar. To show more columns:", "matching_document_document_id": "06f1e751-0ab0-4629-820a-e1bc5bbb114c", "query_weight": 0.8825806451612903 } ]