W E K A

5.1

• NeuralMesh™ by WEKA documentation: Version 5.1
• Documentation revision history
• Introduction: Discover how NeuralMesh delivers a software-only, high-performance, container-native storage system built for AI and data-intensive workloads at scale.
• Cluster capacity and redundancy management
• Filesystems, object stores, and filesystem groups: Learn how filesystems, object stores, and filesystem groups work together to organize storage capacity, tiering, and policy in a WEKA cluster.
• Cluster and filesystem capacity counter definitions: Understand the key capacity counters for effective storage management, including space utilization, availability, and data reduction across SSDs and object storage.
• Networking: Explore network technologies in WEKA, including DPDK, SR-IOV, CPU-optimized networking, UDP mode, high availability, and RDMA/GPUDirect Storage, with configuration guidelines.
• Data lifecycle management overview: Explore the principles for data lifecycle management and how data storage is managed in SSD-only and tiered WEKA system configurations.
• WEKA client and mount modes: Understanding the WEKA system client and possible mount modes of operation in relation to the page cache.
• Cluster architecture overview: Overview of container-based architecture, where interconnected processes within server-hosted containers provide scalable and resilient storage services in a cluster.
• Glossary
• Prerequisites and compatibility: This page describes the prerequisites and compatibility for the installation of the WEKA system.
• System installation on bare metal servers: This topic provides an overview of the automated tools and workflow paths for installing and configuring the WEKA software on a group of bare metal servers (on-premises environment).
• Plan system hardware requirements
• Obtain the installation packages: This page describes registering to get.weka.io and obtaining the WEKA installation packages: WMS, WSA, and WEKA software release.
• Install the cluster using the WMS with WSA
• Install the cluster using the WSA: The WSA (WEKA Software Appliance) is an alternative method to install WEKA software on bare-metal servers. The WSA simplifies and accelerates the installation.
• Manually install OS and system on servers: This requirement only applies when manually preparing and installing the WEKA cluster on bare metal servers.
• Safe server shutdown: Explore the systemd-based mechanism that allows a server to shut down safely. This feature is particularly valuable for converged servers that users might terminate abruptly.
• Manually prepare the system for configuration: If the system is not prepared using the WMS, perform this procedure to set the networking and other tasks before configuring the WEKA cluster.
• Broadcom adapter setup for WEKA clients
• Enable the SR-IOV
• Configure the cluster using the WEKA Configurator: Detailed workflow for WEKA cluster installation in a multi-container backend architecture using the WEKA Configurator.
• Manually configure the cluster using the resources generator: Detailed workflow for manually configuring the WEKA cluster using the resource generator in a multi-container backend architecture.
• VLAN tagging in the WEKA system: WEKA tenant clusters use VLAN tagging to enable isolated network communication between clusters and their clients.
• Perform post-configuration procedures
• Add clients to a bare-metal cluster: This page describes how to add clients to a bare-metal cluster.
• Cloud Deployment Manager Web (CDM Web) User Guide
• WEKA installation on AWS: This section provides detailed instructions on installing a WEKA system on AWS.
• Terraform-AWS-WEKA module description
• Deployment on AWS using Terraform
• Required services and supported regions
• Supported EC2 instance types using Terraform
• WEKA cluster auto-scaling in AWS
• Detailed deployment tutorial: WEKA on AWS using Terraform
• WEKA installation on Azure: This section aims at a system engineer familiar with the Azure fundamentals and experienced in using Terraform to deploy a system on Azure.
• Azure-WEKA deployment Terraform package description
• Deployment on Azure using Terraform
• Required services and supported regions
• Supported virtual machine types
• Auto-scale virtual machines in Azure
• Add clients to a WEKA cluster on Azure
• Troubleshooting
• Detailed deployment tutorial: WEKA on Azure using Terraform: This guide provides step-by-step instructions for deploying the WEKA Data Platform on Microsoft Azure using Terraform, tailored for customers, partners, and WEKA teams.
• WEKA installation on GCP: This section aims at a system engineer familiar with the GCP concepts and experienced in using Terraform to deploy a system on GCP.
• WEKA project description
• GCP-WEKA deployment Terraform package description
• Deployment on GCP using Terraform
• Required services and supported regions
• Supported machine types and storage
• Auto-scale instances in GCP
• Add clients to a WEKA cluster on GCP
• Troubleshooting
• Detailed deployment tutorial: WEKA on GCP using Terraform: This guide provides step-by-step instructions for deploying the WEKA Data Platform on GCP using Terraform, tailored for customers, partners, and WEKA teams.
• Google Kubernetes Engine and WEKA over POSIX deployment: A step-by-step guide to setting up Google Kubernetes Engine (GKE) with WEKA on Google Cloud Platform (GCP), enhancing storage and scalability for demanding Kubernetes workloads.
• WEKA installation on OCI: WEKA Data Platform deployment on Oracle Cloud Infrastructure (OCI) follows bare-metal installation with cloud-specific considerations.
• NeuralMesh Axon overview: Explore the high-performance, co-located architecture and unified namespace model of the NeuralMesh Axon storage solution.
• NeuralMesh Axon deployment: Deploy NeuralMesh Axon and integrate with orchestration services like Kubernetes and SLURM. For partners and skilled customers performing independent installations.
• NeuralMesh Axon maintenance: Manage NeuralMesh Axon performance: replace failed drives and perform server maintenance tasks like reboots.
• Manage the system using the WEKA GUI: WEKA GUI application enables you to configure, administer, and monitor the WEKA system. This page provides an overview of the primary operations, access to the GUI, and system dashboard.
• Manage the system using the WEKA CLI: The overview of the WEKA CLI includes top-level commands, command hierarchy, how to connect to another server, auto-completion, and how to check the status of the cluster.
• WEKA CLI hierarchy: Explore the hierarchical structure of WEKA Command-Line Interface (CLI) commands for easy reference.
• CLI reference guide: This CLI reference guide is generated from the output of running the weka command with the help option. It provides detailed descriptions of available commands, arguments, and options.
• Perform a basic IO sanity check: Use this procedure to perform a basic input/output (IO) sanity check on a newly installed WEKA cluster to confirm it is operational.
• Getting started with WEKA REST API
• WEKA REST API and equivalent CLI commands
• System performance tests: Measure the key performance metrics of a storage system—latency, IOPS, and bandwidth—using standardized testing procedures.
• Performance test environment configurations: This section details the specific hardware and software configurations used to generate the example performance results.
• Manage object stores: This page provides an overview about managing object stores.
• Manage object stores using the GUI: This page describes how to view and manage object stores using the GUI.
• Manage object stores using the CLI: This page describes how to view and manage object stores using the CLI.
• Manage filesystem groups: This page provides an overview about managing filesystem groups.
• Manage filesystem groups using the GUI: This page describes how to view and manage filesystem groups using the GUI.
• Manage filesystem groups using the CLI: This page describes how to view and manage filesystem groups using the CLI.
• Manage filesystems: Filesystem management is an integral part of the successful running and performance of the WEKA system and of overall data lifecycle management.
• Manage filesystems using the GUI: This page describes how to view and manage filesystems using the GUI.
• Manage filesystems using the CLI: This page describes how to view and manage filesystems using the CLI.
• Attach or detach object store buckets: This page describes how to attach or detach object stores buckets to or from filesystems.
• Attach or detach object store bucket using the GUI: This page describes how to attach or detach object stores buckets to or from filesystems using the GUI.
• Attach or detach object store buckets using the CLI: This page describes how to attach or detach object store buckets to or from filesystems using the CLI.
• Manage data lifecycle for tiered systems: Learn how the system manages data storage in tiered configurations, covering lifecycle policies, capacity management, and operational controls.
• Mount filesystems: Discover the two modes for mounting a filesystem on a cluster server: persistent mount mode (stateful) and stateless mount mode. You can also use fstab or autofs for mounting.
• Mount filesystems from Single Client to Multiple Clusters (SCMC): Mount a single stateless WEKA client to multiple clusters simultaneously, optimizing data access and workload distribution.
• Manage authentication across multiple clusters with connection profiles: Learn how to manage authentication across multiple clusters in the WEKA CLI using connection profiles, enabling seamless switching between clusters without re-authentication.
• Snapshots: Snapshots enable the saving of a filesystem state to a directory and can be used for backup, archiving and testing purposes.
• Manage snapshots using the GUI: This page describes how to manage snapshots using the GUI.
• Manage snapshots using the CLI: This page describes how to manage snapshots using the CLI.
• Snap-To-Object: Explore the Snap-To-Object feature, a capability facilitating the seamless data transfer from a designated snapshot to an object store.
• Manage Snap-To-Object using the GUI: This page describes the Snap-To-Object feature, which enables the committing of all the data of a specific snapshot to an object store.
• Manage Snap-To-Object using the CLI: The Snap-To-Object feature enables the committing of all the data of a specific snapshot to an object store.
• Snapshot policies: Snapshot policies define rules and schedules for creating and managing point-in-time data copies, ensuring reliable recovery from deletion, corruption, or integrity issues.
• Manage snapshot policies using the GUI: Manage snapshot policies using the GUI, ensuring efficient data protection.
• Manage snapshot policies using the CLI: Manage snapshot policies using the CLI, ensuring efficient data protection and disaster recovery.
• Quota management: Implement quota management to monitor and control usage of the WEKA filesystem effectively.
• Manage quotas using the GUI: Manage directory quotas and default quota settings for your filesystems using the WEKA GUI.
• Manage quotas using the CLI: Manage directory, user, and group quotas for your filesystems using the WEKA CLI.
• Data catalog: Manage, index, and query filesystem metadata using the data catalog. This feature provides macro-level insights and granular data discovery, enabling visibility into large filesystems at scale.
• Configure data catalog: Deploy and configure data catalog to enable high-performance data indexing and metadata management across filesystems.
• Analyze storage distribution: Analyze storage distribution with the data catalog. Explore directory hierarchies, visualize file statistics, and perform granular metadata discovery.
• Additional protocol containers
• Manage the NFS protocol: The WEKA system enables file access through the NFS protocol instead of the WEKA client.
• Supported NFS client mount parameters: Configure mandatory and recommended NFS client mount options with WEKA based on real-world testing and validation for optimal performance and reliability.
• Manage NFS networking using the GUI: This page describes how to configure the NFS networking using the GUI.
• Manage NFS networking using the CLI: This page describes how to configure the NFS networking using the CLI.
• Manage the S3 protocol: The WEKA configuration of the S3 protocol.
• S3 cluster management
• Manage the S3 service using the GUI: This page describes how to set up, update, monitor, and delete an S3 cluster using the GUI.
• Manage the S3 service using the CLI: This page describes how to set up, update, monitor, and delete an S3 cluster using the GUI.
• S3 buckets management: Explore managing your buckets in WEKA using S3 API, GUI, CLI, or API, with flexible permissions through IAM or bucket policies for anonymous access.
• Manage S3 buckets using the GUI: This page describes how to manage S3 buckets using the GUI.
• Manage S3 buckets using the CLI: This page describes how to manage S3 buckets using the CLI.
• S3 users and authentication: Manage access to the S3 API using local WEKA accounts or LDAP-integrated identities. Local users require the S3 role and an attached IAM policy.
• Manage S3 users and authentication using the CLI: This page describes how to gain and obtain access permissions to the S3 protocol using the CLI.
• Manage S3 service accounts using the CLI: This page describes how to add and control S3 service accounts using the CLI.
• Configure S3 LDAP authentication: Integrate S3 service authentication with existing LDAP to automate key generation and centralize identity management.
• Configure S3 OIDC authentication: Enable users and applications to authenticate with your IdP and receive temporary S3 credentials, eliminating the need for static access keys.
• S3 lifecycle rules management: Explore how S3 lifecycle rules automate object expiration. These rules apply to the underlying data, ensuring changes reflect across all access protocols.
• Manage S3 lifecycle rules using the GUI: This page describes how to manage information lifecycle (ILM) rules for S3 buckets using the GUI.
• Manage S3 lifecycle rules using the CLI: This page describes how to manage information lifecycle (ILM) rules for S3 buckets using the CLI.
• Audit S3 APIs: This page describes how to set up an HTTP webhook for S3 audit purposes.
• Configure audit webhook using the GUI
• Configure audit webhook using the CLI: This page describes how to set up an HTTP webhook for S3 audit purposes using the CLI.
• Example: How to use Splunk to audit S3: This page describes an example for using Splunk to audit S3.
• Example: How to use S3 audit events for tracking and security: Learn how to interpret and use an S3 audit event generated by the WEKA S3 system.
• S3 supported APIs and limitations: This page describes limitations concerning the S3 service and protocol implementation.
• S3 examples using boto3: This page provides some examples of using the S3 API.
• Configure and use AWS CLI with WEKA S3 storage: Learn how to configure and use the AWS CLI with WEKA S3 storage.
• S3 bucket notifications: Learn to automate workflows by receiving real-time notifications for specific events in S3 buckets.
• Configure S3 Performance Buckets: Learn how to configure global defaults and individual bucket settings for S3 Performance Buckets to optimize S3 operations.
• Manage the SMB protocol: Configure and control WEKA's SMB-W implementation for seamless cross-platform file access across Windows, Linux, and macOS clients with concurrent multi-protocol support.
• Manage SMB using the GUI: Use the GUI to configure an SMB-W cluster on WEKA filesystems and manage cluster settings, shares, and Active Directory integration.
• Manage SMB using the CLI: Use the GUI to configure an SMB-W cluster on WEKA filesystems and manage cluster settings, shares, and Active Directory integration.
• Security overview: The system implements a comprehensive security framework with multiple controls to ensure secure communication and protect user data in the cluster.
• Obtain authentication tokens
• Manage token expiration: Learn how to configure and manage token expiration settings to maintain a secure authentication environment aligned with best practices.
• Manage account lockout threshold policy
• Manage KMS: Efficiently manage and safeguard WEKA system keys through strategic KMS configurations and best practices. Optimize security and operational resilience.
• Manage KMS using GUI: Explore procedures for managing Key Management System (KMS) integration with the WEKA system using the GUI.
• Manage KMS using CLI: Explore commands for managing Key Management System (KMS) integration with the WEKA system using the CLI.
• Manage TLS certificates: Manage TLS certificates to encrypt data in transit and verify system identity, ensuring secure communication in the WEKA cluster.
• Manage TLS certificates using GUI
• Manage TLS certificates using CLI
• Manage Cross-Origin Resource Sharing
• Manage CIDR-based security policies: Manage CIDR-based security policies to control access to WEKA clusters based on client IP address ranges, enhancing security and simplifying administration.
• Manage login banner: This page describes how to set a login banner displayed on the sign-in page.
• Secure cluster membership with join secret authentication
• Security hardening: Harden the security of a WEKA deployment. This guide covers Linux best practices, architectural requirements, port configurations, and auditing capabilities.
• Licensing overview: Understand WEKA license types, capacity rules, and status checks.
• Apply a classic license: Obtain, apply, reuse, and resize a classic WEKA license for a cluster.
• Alerts: Explore how alerts identify problematic, ongoing states within a cluster that require administrative attention. These notifications provide, actionable insights into system health and performance.
• Manage alerts using the GUI: View, mute, and unmute alerts to maintain cluster health.
• Manage alerts using the CLI: This page describes how to manage alerts using the CLI.
• List of alerts and corrective actions: Check WEKA system alerts and take necessary actions based on severity and nature.
• Events: WEKA system events provide timestamped information about cluster operations and changes.
• Manage events using the GUI: This page describes how to manage events using the GUI.
• Manage events using the CLI: This page describes how to manage events using the CLI.
• Events list: Explore the various events the WEKA system produces, organized according to their respective categories.
• Statistics: This page describes the statistics available in the WEKA system and how to work with them.
• Manage statistics using the GUI: This page describes how to manage the statistics using the GUI.
• Manage statistics using the CLI: This page describes how to manage the statistics using the CLI.
• Statistics list: Explore the various statistics the WEKA system produces, organized according to their respective category labels.
• Insights
• System congestion: This page describes possible congestion issues in the WEKA system.
• User management: The WEKA system enables managing user access and roles locally and through organizational directories like LDAP or AD. This topic covers user types, authentication methods, and management.
• Manage users using the GUI: Explore the GUI to manage local users and configure the user directories LDAP or AD.
• Manage users using the CLI: Explore the CLI to perform a broader range of user management tasks, including creating, updating, and deleting local users, managing access, and authenticating users through the LDAP or AD directory.
• WEKA native multi-tenancy management: Logically separate users and resources on one WEKA cluster. Host tenants with independent LDAP, KMS, and QoS. Scale from small workloads to multi-petabyte deployments on shared infrastructure.
• Multi-tenancy cluster-level administration: Manage cluster-level multi-tenancy by configuring network spaces and isolated tenant environments to define resource quotas and security policies.
• Multi-tenancy tenant-level administration: Manage isolated storage, users, and security within a specific tenant boundary.
• Expand and shrink cluster resources: Expand and shrink a cluster in a homogeneous WEKA system configuration.
• Add a backend server
• Expand specific resources of a container: Guidelines for expansion processes that only involve the addition of a specific resource.
• Shrink a cluster
• Container state and status fields: Learn how to interpret how state and status in container and process output from the CLI and REST API.
• Background tasks: This page describes the management of background tasks running on the WEKA system.‌
• Set up a Data Services container for background tasks: Efficiently manage resource-intensive tasks with at least one Data Services container for improved performance and reliability.
• Manage background tasks using the GUI
• Manage background tasks using the CLI
• Audit and forwarding management: Manage the forwarding of data access audit events to external monitoring systems.
• Manage audit and forwarding using the GUI: Configure cluster-wide audit settings and manage telemetry exports to forward audit data to external systems using the NeuralMesh GUI.
• Manage audit and forwarding using the CLI
• Upgrade WEKA versions: Upgrade your WEKA system with the latest version.
• Manage WEKA drivers: Learn how to manage WEKA drivers to ensure they are properly built, installed, signed, and distributed for the appropriate kernel version.
• Drivers distribution service: A centralized service that delivers pre-built WEKA kernel drivers for supported Linux distributions, eliminating manual compilation while ensuring consistent deployment across environments.
• Drives sharing: Share access to physical NVMe devices among multiple Drive IO processes through a centralized SSD Proxy process by dividing a single physical drive into multiple virtual drives.
• Deploy monitoring tools using the WEKA Management Station (WMS): Deploy the monitoring tools, LWH, WEKAmon, and SnapTool, using the WEKA Management Station (WMS) in an on-premises environment.
• WEKA Home - The WEKA support cloud: Improve the WEKA support process with WEKA Home.
• Local WEKA Home overview: Local WEKA Home is a private instance of WEKA Home deployed with a WEKA cluster in a private network.
• Deploy Local WEKA Home on K8s: Manage the deployment, upgrade, and maintenance of Local WEKA Home (LWH) on Kubernetes (K8s) cluster. This deployment method provides a scalable, on-premises observability solution for WEKA clusters.
• Manage remote debugging sessions on K8s: Configure and control secure, real-time access for WEKA Customer Success Team to troubleshoot cluster issues using charts/wekahome/values.yaml on K8s deployment.
• Deploy Local WEKA Home on K3s: Manage the deployment, upgrade, and maintenance of Local WEKA Home (LWH) on K3s. This single-node architecture provides an on-premises observability solution for WEKA clusters.
• Manage remote debugging sessions on K3s: Configure and control secure, real-time access for WEKA Customer Success Team to troubleshoot cluster issues using homecli on K3s deployment.
• LWH stats: sizing and performance optimization: Size and tune LWH stats components for high throughput by configuring resource allocations based on stats. This prevents stats stream saturation, processing delays, and NATS backpressure.
• Deploy Local WEKA Home on Minikube: Learn how to deploy, upgrade, configure, and troubleshoot Local WEKA Home v2x on Minikube, an on-premises, observability solution for WEKA clusters.
• Explore cluster insights
• Explore performance statistics in Grafana
• Manage alerts and integrations
• Enforce security and compliance
• Optimize support and data management
• Export cluster metrics to Prometheus: Export a set of WEKA cluster metrics from CWH directly into Prometheus without using WEKAmon.
• NeuralMesh Observe overview: Validate performance, monitor cluster health, and troubleshoot your WEKA estate from a single SaaS observability interface.
• Set up WEKAmon for external monitoring: Configure WEKAmon to integrate Grafana and Prometheus for centralized monitoring of WEKA cluster metrics, logs, alerts, and statistics.
• Monitor WEKA clusters in Kubernetes with Prometheus and Grafana: Deploy Prometheus and Grafana to monitor the health and performance of WEKA clusters in Kubernetes.
• Composable clusters for multi-tenancy in Kubernetes: Explore WEKA’s multi-tenancy with composable clusters, ensuring full resource isolation and optimal performance through Kubernetes-driven deployment.
• WEKA Operator deployments: Discover how the WEKA Operator streamlines deploying, scaling, and managing the WEKA Data Platform on Kubernetes, delivering high-performance storage for compute-intensive workloads like AI and HPC.
• Driver management with the WEKA Operator: Automate driver lifecycle through WEKA Operator. Source pre-built or local drivers to match kernels and load automatically.
• Set up protocols on K8s with WEKA Operator: Configure S3, NFS-W, and SMB-W protocols using the WEKA Kubernetes operator to provide object and file access to filesystems.
• Upgrade protocol containers on the WEKA Operator: Upgrade NFS, S3, and SMB-W protocol containers on a Kubernetes cluster managed by the WEKA Operator using version-based node labels and the roleNodeSelector field on the WekaCluster custom resource.
• Set up audit logs on K8s with WEKA Operator: Configure audit log exports to external destinations using the WEKA Kubernetes operator.
• Deploy WEKA on managed Kubernetes services: Deploy the WEKA operator on a cloud-managed Kubernetes service to mount WEKA filesystems from application pods running in a managed cluster.
• Migrate standalone CSI to WEKA Operator-embedded: Migrate the CSI plugin to the WEKA Operator to centralize cluster management and streamline future updates.
• Weka Operator secrets management: Manage the Kubernetes secrets created by the Weka operator to store credentials and connection information required for cluster interaction. The operator automatically handles the lifecycle of these s
• WEKA Operator operations: Manage hardware, scale clusters, and optimize resources to ensure system stability and performance.
• WEKApod Data Platform Appliance overview
• WEKApod servers overview
• Rack installation
• WEKApod initial system setup and configuration
• WEKApod support process
• Amazon SageMaker HyperPod and WEKA Integrations
• Deploy a new Amazon SageMaker HyperPod cluster with WEKA
• Add WEKA to an existing Amazon SageMaker HyperPod cluster
• AWS ParallelCluster and WEKA Integration
• Azure CycleCloud for SLURM and WEKA Integration: Learn to integrate Azure CycleCloud with the WEKA Data Platform and SLURM scheduler to streamline HPC cluster management and enable high-performance, scalable data solutions for AI, ML, and analytics.
• WEKA and Slurm integration: Explore the architecture and configuration of an HPC cluster using the Slurm workload manager for job scheduling and WEKA as the high-performance data platform.
• Manage CPU allocations for WEKA and Slurm
• Best practices for WEKA stateless client and Kubernetes: Use the WEKA Operator for WEKA CSI in Kubernetes. If using stateless clients, follow practices to reserve CPU, prevent contention, and ensure stable IO.
• Storage expansion best practice: Explore the best practices for expanding WEKA clusters, focusing on maintaining optimal performance in heterogeneous environments.
• Maximize GPU infrastructure resilience with NeuralMesh AXON: Deliver enterprise-grade reliability in converged GPU environments with failure domains. This approach keeps data protected and accessible, even when high-performance apps cause frequent reboots.
• Release support and commitments: Learn about the WEKA software release cycle, support timelines, and upgrade policies to maintain system reliability and security.
• Get support for your WEKA system: Get help from the WEKA Customer Success Team, choose the right severity level, and prepare the information needed for faster resolution.
• Diagnostics management
• Traces management: This page describes how to manage traces generated by the WEKA processes.
• Manage traces using the GUI
• Manage traces using the CLI
• Protocols debug level management: This page describes the S3, NFS, and SMB protocols debug level management. A higher debug level provides more details for investigating issues.
• Manage protocols debug level using the GUI
• Manage protocols debug level using the CLI
• Diagnostics data management: Manage diagnostics data for clusters, servers, and containers with CLI commands.
• WEKA CSI Plugin: Connect Kubernetes worker nodes to the WEKA data platform to leverage its capabilities.
• Deployment
• Storage class configurations
• Configure storage class mount options: Configure the WEKA CSI Plugin to use specific mount options for tailored storage control.
• Dynamic and static provisioning
• Launch an application using WEKA as the POD's storage
• Configure topology-aware volume provisioning: Learn how to use the Kubernetes Container Storage Interface (CSI) topology feature to control where volumes are provisioned based on availability zones.
• Add SELinux support
• NFS transport failback
• Upgrade legacy persistent volumes for capacity enforcement
• Troubleshooting
• Create a Volume Snapshot: Learn how to create point-in-time snapshots of WEKA Kubernetes volumes using the CSI snapshot capability.
• Create a client image
• Update WMS and WSA: Learn to manually update the WEKA Management Station (WMS) and WEKA Software Appliance (WSA) on sites with Internet access and dark sites to maintain system security and functionality.
• BIOS tool: This tool simplifies managing BIOS settings across multiple servers, ensuring consistency and reducing manual configuration efforts.

5.0

• NeuralMesh™ by WEKA documentation: Version 5.0
• Documentation revision history
• Introduction: Discover how NeuralMesh delivers a software-only, high-performance, container-native storage system built for AI and data-intensive workloads at scale.
• Cluster capacity and redundancy management
• Filesystems, object stores, and filesystem groups: This page describes the three entity types relevant to data storage in the WEKA system.
• Cluster and filesystem capacity counter definitions: Understand the key capacity counters for effective storage management, including space utilization, availability, and data reduction across SSDs and object storage.
• Networking: Explore network technologies in WEKA, including DPDK, SR-IOV, CPU-optimized networking, UDP mode, high availability, and RDMA/GPUDirect Storage, with configuration guidelines.
• Data lifecycle management overview: Explore the principles for data lifecycle management and how data storage is managed in SSD-only and tiered WEKA system configurations.
• WEKA client and mount modes: Understanding the WEKA system client and possible mount modes of operation in relation to the page cache.
• Cluster architecture overview: Overview of WEKA's container-based architecture, where interconnected processes within server-hosted containers provide scalable and resilient storage services in a cluster.
• Glossary
• Prerequisites and compatibility: This page describes the prerequisites and compatibility for the installation of the WEKA system.
• System installation on bare metal servers: This topic provides an overview of the automated tools and workflow paths for installing and configuring the WEKA software on a group of bare metal servers (on-premises environment).
• Plan the WEKA system hardware requirements
• Obtain the WEKA installation packages: This page describes registering to get.weka.io and obtaining the WEKA installation packages: WMS, WSA, and WEKA software release.
• Install the WEKA cluster using the WMS with WSA
• Install the WEKA cluster using the WSA: The WSA (WEKA Software Appliance) is an alternative method to install WEKA software on bare-metal servers. The WSA simplifies and accelerates the installation.
• Manually install OS and WEKA on servers: This requirement only applies when manually preparing and installing the WEKA cluster on bare metal servers.
• Safe server shutdown: Explore the systemd-based mechanism that allows a server to shut down safely. This feature is particularly valuable for converged servers that users might terminate abruptly.
• Manually prepare the system for WEKA configuration: If the system is not prepared using the WMS, perform this procedure to set the networking and other tasks before configuring the WEKA cluster.
• Broadcom adapter setup for WEKA clients
• Enable the SR-IOV
• Configure the WEKA cluster using the WEKA Configurator: Detailed workflow for WEKA cluster installation in a multi-container backend architecture using the WEKA Configurator.
• Manually configure the WEKA cluster using the resources generator: Detailed workflow for manually configuring the WEKA cluster using the resource generator in a multi-container backend architecture.
• VLAN tagging in the WEKA system: WEKA tenant clusters use VLAN tagging to enable isolated network communication between clusters and their clients.
• Perform post-configuration procedures
• Add clients to an on-premises WEKA cluster: This page describes how to add clients to a bare-metal cluster.
• Cloud Deployment Manager Web (CDM Web) User Guide
• Cloud Deployment Manager Local (CDM Local) User Guide
• System installation on AWS: This section provides detailed instructions on installing a WEKA system on AWS.
• Terraform-AWS-WEKA module description
• Deployment on AWS using Terraform
• Required services and supported regions
• Supported EC2 instance types using Terraform
• WEKA cluster auto-scaling in AWS
• Detailed deployment tutorial: WEKA on AWS using Terraform
• System installation on Azure: This section aims at a system engineer familiar with the Azure fundamentals and experienced in using Terraform to deploy a system on Azure.
• Azure-WEKA deployment Terraform package description
• Deployment on Azure using Terraform
• Required services and supported regions
• Supported virtual machine types
• Auto-scale virtual machines in Azure
• Add clients to a WEKA cluster on Azure
• Troubleshooting
• Detailed deployment tutorial: WEKA on Azure using Terraform: This guide provides step-by-step instructions for deploying the WEKA Data Platform on Microsoft Azure using Terraform, tailored for customers, partners, and WEKA teams.
• System installation on GCP: This section aims at a system engineer familiar with the GCP concepts and experienced in using Terraform to deploy a system on GCP.
• WEKA project description
• GCP-WEKA deployment Terraform package description
• Deployment on GCP using Terraform
• Required services and supported regions
• Supported machine types and storage
• Auto-scale instances in GCP
• Add clients to a WEKA cluster on GCP
• Troubleshooting
• Detailed deployment tutorial: WEKA on GCP using Terraform: This guide provides step-by-step instructions for deploying the WEKA Data Platform on GCP using Terraform, tailored for customers, partners, and WEKA teams.
• Google Kubernetes Engine and WEKA over POSIX deployment: A step-by-step guide to setting up Google Kubernetes Engine (GKE) with WEKA on Google Cloud Platform (GCP), enhancing storage and scalability for demanding Kubernetes workloads.
• System installation on OCI: WEKA Data Platform deployment on Oracle Cloud Infrastructure (OCI) follows bare-metal installation with cloud-specific considerations.
• NeuralMesh Axon overview: Explore the high-performance, co-located architecture and unified namespace model of the NeuralMesh Axon storage solution.
• NeuralMesh Axon deployment: Deploy NeuralMesh Axon and integrate with orchestration services like Kubernetes and SLURM. For partners and skilled customers performing independent installations.
• NeuralMesh Axon maintenance: Manage NeuralMesh Axon performance: replace failed drives and perform server maintenance tasks like reboots.
• Manage the system using the GUI: WEKA GUI application enables you to configure, administer, and monitor the WEKA system. This page provides an overview of the primary operations, access to the GUI, and system dashboard.
• Manage the system using the CLI: The overview of the WEKA CLI includes top-level commands, command hierarchy, how to connect to another server, auto-completion, and how to check the status of the cluster.
• WEKA CLI hierarchy: Explore the hierarchical structure of WEKA Command-Line Interface (CLI) commands for easy reference.
• CLI reference guide: This CLI reference guide is generated from the output of running the weka command with the help option. It provides detailed descriptions of available commands, arguments, and options.
• Perform a basic IO sanity check: Use this procedure to perform a basic input/output (IO) sanity check on a newly installed WEKA cluster to confirm it is operational.
• Getting started with REST API
• REST API and equivalent CLI commands
• System performance tests: Measure the key performance metrics of a storage system—latency, IOPS, and bandwidth—using standardized testing procedures.
• Performance test environment configurations: This section details the specific hardware and software configurations used to generate the example performance results.
• Manage object stores: This page provides an overview about managing object stores.
• Manage object stores using the GUI: This page describes how to view and manage object stores using the GUI.
• Manage object stores using the CLI: This page describes how to view and manage object stores using the CLI.
• Manage filesystem groups: Configure centralized tiering policies to enforce data lifecycle rules across multiple filesystems.
• Manage filesystem groups using the GUI: Configure filesystem groups and tiering policies using the GUI.
• Manage filesystem groups using the CLI: Configure filesystem groups and tiering policies using the CLI.
• Manage filesystems: Filesystem management is an integral part of the successful running and performance of the WEKA system and of overall data lifecycle management.
• Manage filesystems using the GUI: This page describes how to view and manage filesystems using the GUI.
• Manage filesystems using the CLI: This page describes how to view and manage filesystems using the CLI.
• Attach or detach object store buckets: This page describes how to attach or detach object stores buckets to or from filesystems.
• Attach or detach object store bucket using the GUI: This page describes how to attach or detach object stores buckets to or from filesystems using the GUI.
• Attach or detach object store buckets using the CLI: This page describes how to attach or detach object store buckets to or from filesystems using the CLI.
• Manage data lifecycle for tiered systems: Learn how the system manages data storage in tiered configurations, covering lifecycle policies, capacity management, and operational controls.
• Mount filesystems: Discover the two modes for mounting a filesystem on a cluster server: persistent mount mode (stateful) and stateless mount mode. You can also use fstab or autofs for mounting.
• Mount filesystems from Single Client to Multiple Clusters (SCMC): Mount a single stateless WEKA client to multiple clusters simultaneously, optimizing data access and workload distribution.
• Manage authentication across multiple clusters with connection profiles: Learn how to manage authentication across multiple clusters in the WEKA CLI using connection profiles, enabling seamless switching between clusters without re-authentication.
• Snapshots: Snapshots enable the saving of a filesystem state to a directory and can be used for backup, archiving and testing purposes.
• Manage snapshots using the GUI: This page describes how to manage snapshots using the GUI.
• Manage snapshots using the CLI: This page describes how to manage snapshots using the CLI.
• Snap-To-Object: Explore the Snap-To-Object feature, a capability facilitating the seamless data transfer from a designated snapshot to an object store.
• Manage Snap-To-Object using the GUI: This page describes the Snap-To-Object feature, which enables the committing of all the data of a specific snapshot to an object store.
• Manage Snap-To-Object using the CLI: The Snap-To-Object feature enables the committing of all the data of a specific snapshot to an object store.
• Snapshot policies: Snapshot policies define rules and schedules for creating and managing point-in-time data copies, ensuring reliable recovery from deletion, corruption, or integrity issues.
• Manage snapshot policies using the GUI: Manage snapshot policies using the GUI, ensuring efficient data protection.
• Manage snapshot policies using the CLI: Manage snapshot policies using the CLI, ensuring efficient data protection and disaster recovery.
• Quota management: Implement quota management to monitor and control usage of the WEKA filesystem effectively.
• Manage quotas using the GUI: This page describes how to manage quotas using the GUI.
• Manage quotas using the CLI: This page describes how to manage quotas using the CLI.
• Additional protocol containers
• Manage the NFS protocol: The WEKA system enables file access through the NFS protocol instead of the WEKA client.
• Supported NFS client mount parameters: Configure mandatory and recommended NFS client mount options with WEKA based on real-world testing and validation for optimal performance and reliability.
• Manage NFS networking using the GUI: This page describes how to configure the NFS networking using the GUI.
• Manage NFS networking using the CLI: This page describes how to configure the NFS networking using the CLI.
• Manage the S3 protocol: The WEKA configuration of the S3 protocol.
• S3 cluster management
• Manage the S3 service using the GUI: This page describes how to set up, update, monitor, and delete an S3 cluster using the GUI.
• Manage the S3 service using the CLI: This page describes how to set up, update, monitor, and delete an S3 cluster using the GUI.
• S3 buckets management: Explore managing your buckets in WEKA using S3 API, GUI, CLI, or API, with flexible permissions through IAM or bucket policies for anonymous access.
• Manage S3 buckets using the GUI: This page describes how to manage S3 buckets using the GUI.
• Manage S3 buckets using the CLI: This page describes how to manage S3 buckets using the CLI.
• S3 users and authentication: This page describes how to gain and obtain access permissions to the S3 protocol.
• Manage S3 users and authentication using the CLI: This page describes how to gain and obtain access permissions to the S3 protocol using the CLI.
• Manage S3 service accounts using the CLI: This page describes how to add and control S3 service accounts using the CLI.
• S3 lifecycle rules management: S3 lifecycle rules management in WEKA automates object organization and expiration in S3 buckets through customizable rules, ensuring consistent application across all access protocols.
• Manage S3 lifecycle rules using the GUI: This page describes how to manage information lifecycle (ILM) rules for S3 buckets using the GUI.
• Manage S3 lifecycle rules using the CLI: This page describes how to manage information lifecycle (ILM) rules for S3 buckets using the CLI.
• Audit S3 APIs: This page describes how to set up an HTTP webhook for S3 audit purposes.
• Configure audit webhook using the GUI
• Configure audit webhook using the CLI: This page describes how to set up an HTTP webhook for S3 audit purposes using the CLI.
• Example: How to use Splunk to audit S3: This page describes an example for using Splunk to audit S3.
• Example: How to use S3 audit events for tracking and security: Learn how to interpret and use an S3 audit event generated by the WEKA S3 system.
• S3 supported APIs and limitations: This page describes limitations concerning the S3 service and protocol implementation.
• S3 examples using boto3: This page provides some examples of using the S3 API.
• Configure and use AWS CLI with WEKA S3 storage: Learn how to configure and use the AWS CLI with WEKA S3 storage.
• S3 bucket notifications: Learn to automate workflows by receiving real-time notifications for specific events in S3 buckets.
• Configure S3 Performance Buckets: Learn how to configure global defaults and individual bucket settings for S3 Performance Buckets to optimize S3 operations.
• Manage the SMB protocol: Configure and control WEKA's SMB-W implementation for seamless cross-platform file access across Windows, Linux, and macOS clients with concurrent multi-protocol support.
• Manage SMB using the GUI: Use the GUI to configure an SMB-W cluster on WEKA filesystems and manage cluster settings, shares, and Active Directory integration.
• Manage SMB using the CLI: Use the GUI to configure an SMB-W cluster on WEKA filesystems and manage cluster settings, shares, and Active Directory integration.
• System security overview: The system implements a comprehensive security framework with multiple controls to ensure secure communication and protect user data in the cluster.
• Obtain authentication tokens
• Manage token expiration: Learn how to configure and manage token expiration settings to maintain a secure authentication environment aligned with best practices.
• Manage account lockout threshold policy
• Manage KMS: Efficiently manage and safeguard WEKA system keys through strategic KMS configurations and best practices. Optimize security and operational resilience.
• Manage KMS using GUI: Explore procedures for managing Key Management System (KMS) integration with the WEKA system using the GUI.
• Manage KMS using CLI: Explore commands for managing Key Management System (KMS) integration with the WEKA system using the CLI.
• Manage TLS certificates: Manage TLS certificates to encrypt data in transit and verify system identity, ensuring secure communication in the WEKA cluster.
• Manage TLS certificates using GUI
• Manage TLS certificates using CLI
• Manage Cross-Origin Resource Sharing
• Manage CIDR-based security policies: Manage CIDR-based security policies to control access to WEKA clusters based on client IP address ranges, enhancing security and simplifying administration.
• Manage login banner: This page describes how to set a login banner displayed on the sign-in page.
• Secure cluster membership with join secret authentication
• Security hardening: Harden the security of a WEKA deployment. This guide covers Linux best practices, architectural requirements, port configurations, and auditing capabilities.
• License overview: This page describes how licensing works in a WEKA cluster.
• Classic license: Learn how to obtain and apply a classic WEKA license, a time-based license purchased for a predetermined period, to your WEKA cluster.
• Alerts: This page describes the alerts that can be received in this version of the WEKA system.
• Manage alerts using the GUI: This page describes how to manage alerts using the GUI.
• Manage alerts using the CLI: This page describes how to manage alerts using the CLI.
• List of alerts and corrective actions: Check WEKA system alerts and take necessary actions based on severity and nature.
• Events: WEKA system events provide timestamped information about cluster operations and changes.
• Manage events using the GUI: This page describes how to manage events using the GUI.
• Manage events using the CLI: This page describes how to manage events using the CLI.
• Events list: Explore the various events the WEKA system produces, organized according to their respective categories.
• Statistics: This page describes the statistics available in the WEKA system and how to work with them.
• Manage statistics using the GUI: This page describes how to manage the statistics using the GUI.
• Manage statistics using the CLI: This page describes how to manage the statistics using the CLI.
• Statistics list: Explore the various statistics the WEKA system produces, organized according to their respective category labels.
• Insights
• System congestion: This page describes possible congestion issues in the WEKA system.
• User management: The WEKA system enables managing user access and roles locally and through organizational directories like LDAP or AD. This topic covers user types, authentication methods, and management.
• Manage users using the GUI: Explore the GUI to manage local users and configure the user directories LDAP or AD.
• Manage users using the CLI: Explore the CLI to perform a broader range of user management tasks, including creating, updating, and deleting local users, managing access, and authenticating users through the LDAP or AD directory.
• Organizations management: Understand how WEKA supports multi-tenancy by logically separating users and resources using organizations.
• Manage organizations using the GUI: Explore how to manage organizations using the GUI.
• Manage organizations using the CLI: Explore how to manage organizations using the CLI.
• Mount authentication for organization filesystems: Explore the authentication process for users mounting organization filesystems.
• Expand and shrink cluster resources: Expand and shrink a cluster in a homogeneous WEKA system configuration.
• Add a backend server
• Expand specific resources of a container: Guidelines for expansion processes that only involve the addition of a specific resource.
• Shrink a cluster
• Background tasks: This page describes the management of background tasks running on the WEKA system.‌
• Set up a Data Services container for background tasks: Efficiently manage resource-intensive tasks with at least one Data Services container for improved performance and reliability.
• Manage background tasks using the GUI
• Manage background tasks using the CLI
• Audit and forwarding management: Manage the forwarding of data access audit events to external monitoring systems.
• Manage audit and forwarding using the GUI: Configure cluster-wide audit settings and manage telemetry exports to forward audit data to external systems using the NeuralMesh GUI.
• Manage audit and forwarding using the CLI
• Upgrade WEKA versions: Upgrade your WEKA system with the latest version.
• Manage WEKA drivers: Learn how to manage WEKA drivers to ensure they are properly built, installed, signed, and distributed for the appropriate kernel version.
• Drivers distribution service: A centralized service that delivers pre-built WEKA kernel drivers for supported Linux distributions, eliminating manual compilation while ensuring consistent deployment across environments.
• Deploy monitoring tools using the WEKA Management Station (WMS): Deploy the monitoring tools, LWH, and WEKAmon, using the WEKA Management Station (WMS) in an on-premises environment.
• WEKA Home - The WEKA support cloud: Improve the WEKA support process with WEKA Home.
• Local WEKA Home overview: Local WEKA Home is a private instance of WEKA Home deployed with a WEKA cluster in a private network.
• Deploy Local WEKA Home on K8s: Manage the deployment, upgrade, and maintenance of Local WEKA Home (LWH) on Kubernetes (K8s) cluster. This deployment method provides a scalable, on-premises observability solution for WEKA clusters.
• Deploy Local WEKA Home on K3s: Manage the deployment, upgrade, and maintenance of Local WEKA Home (LWH) on K3s. This single-node architecture provides an on-premises observability solution for WEKA clusters.
• Deploy Local WEKA Home on Minikube: Deploy, upgrade, and manage Local WEKA Home (LWH) v2.x on Minikube, an on-premises, observability solution for WEKA clusters.
• Explore cluster insights
• Explore performance statistics in Grafana
• Manage alerts and integrations
• Enforce security and compliance
• Optimize support and data management
• Export cluster metrics to Prometheus: Export a set of WEKA cluster metrics from CWH directly into Prometheus without using WEKAmon.
• Set up WEKAmon for external monitoring: Configure WEKAmon to integrate Grafana and Prometheus for centralized monitoring of WEKA cluster metrics, logs, alerts, and statistics.
• Monitor WEKA clusters in Kubernetes with Prometheus and Grafana: Deploy Prometheus and Grafana to monitor the health and performance of WEKA clusters in Kubernetes.
• Composable clusters for multi-tenancy in Kubernetes: Explore WEKA’s multi-tenancy with composable clusters, ensuring full resource isolation and optimal performance through Kubernetes-driven deployment.
• WEKA Operator deployments: Discover how the WEKA Operator streamlines deploying, scaling, and managing the WEKA Data Platform on Kubernetes, delivering high-performance storage for compute-intensive workloads like AI and HPC.
• Deploy the WEKA client on Amazon EKS: Deploy the WEKA client on an existing Amazon EKS cluster to enable Kubernetes workloads to access the WEKA filesystem.
• WEKA Operator day-2 operations: Manage hardware, scale clusters, and optimize resources to ensure system stability and performance.
• WEKApod Data Platform Appliance overview
• WEKApod servers overview
• Rack installation
• WEKApod initial system setup and configuration
• WEKApod support process
• Amazon SageMaker HyperPod and WEKA Integrations
• Deploy a new Amazon SageMaker HyperPod cluster with WEKA
• Add WEKA to an existing Amazon SageMaker HyperPod cluster
• AWS ParallelCluster and WEKA Integration
• Azure CycleCloud for SLURM and WEKA Integration: Learn to integrate Azure CycleCloud with the WEKA Data Platform and SLURM scheduler to streamline HPC cluster management and enable high-performance, scalable data solutions for AI, ML, and analytics.
• WEKA and Slurm integration: Explore the architecture and configuration of an HPC cluster using the Slurm workload manager for job scheduling and WEKA as the high-performance data platform.
• Manage CPU allocations for WEKA and Slurm
• Best practices for WEKA stateless client and Kubernetes: Use the WEKA Operator for WEKA CSI in Kubernetes. If using stateless clients, follow practices to reserve CPU, prevent contention, and ensure stable IO.
• Storage expansion best practice: Explore the best practices for expanding WEKA clusters, focusing on maintaining optimal performance in heterogeneous environments.
• Release support and commitments: Learn about the WEKA software release cycle, support timelines, and upgrade policies to maintain system reliability and security.
• Get support for your WEKA system: Discover WEKA's support policies, proactive approach, and helpful tips for a seamless start.
• Diagnostics management
• Traces management: This page describes how to manage traces generated by the WEKA processes.
• Manage traces using the GUI
• Manage traces using the CLI
• Protocols debug level management: This page describes the S3, NFS, and SMB protocols debug level management. A higher debug level provides more details for investigating issues.
• Manage protocols debug level using the GUI
• Manage protocols debug level using the CLI
• Diagnostics data management: Manage diagnostics data for clusters, servers, and containers with CLI commands.
• WEKA CSI Plugin: Connect Kubernetes worker nodes to the WEKA data platform to leverage its capabilities.
• Deployment
• Storage class configurations
• Configure storage class mount options: Configure the WEKA CSI Plugin to use specific mount options for tailored storage control.
• Dynamic and static provisioning
• [Launch an application using WEKA as the POD's storage](https://docs.weka.io/5.0/appendices/weka-csi-plugin/l

… [truncated — open the raw llms.txt above for the full file]