Kintone Engineering Blog

Learn about Kintone's engineering efforts. Kintone is provided by Cybozu Inc., a Tokyo-based public company founded in 1997.

Introducing pvc-autoresizer

By Akihiro Ikezoe.

Today, we are excited to announce pvc-autoresizer, which is new, Kubernetes-native, open-source software to provide the ability to expand the size of Persistent Volume Claims (PVCs) on Kubernetes cluster automatically.

pvc-autoresizer can virtually be used with all CSI drivers that support VolumeExpansion.

Why pvc-autoresizer?

To create persistent storage in Kubernetes, we can use Kubernetes resources called Persistent Volume (PV) and Persistent Volume Claim.

If you have CSI drivers which support Dynamic Provisioning, a PV will be created dynamically when a PVC is created. On the other hand, there is no dynamic Volume Expansion feature in Kubernetes.

As the number of volumes increases, extending them one by one can become a burden.


pvc-autoresizer can be used the PVCs that meets the following conditions:

  • Provisioned by CSI drivers which support VolumeExpansion.
  • Filesystem volume mode.

For example:

TopoLVM is a new local storage for Kubernetes. If you're interested in TopoLVM, you can read the following article.

How it works

To expand a PVC, pvc-autoresizer works as follows:

  1. Get the target PVC information from kube-apiserver.
  2. Get StorageClass related to the PVC from kube-apiserver.
  3. Get filesystem usage metrics from Prometheus that scrapes the information from kubelet.
  4. Expand PVC storage request size if PVC has less than the specified amount of free filesystem capacity.

The architecture of pvc-autoresizer
The architecture of pvc-autoresizer

How to use

Learn how to use pvc-autoresizer in a Kubernetes cluster where your target CSI driver is deployed.

Setup Prometheus

pvc-autoresizer collects PV filesystem usage from Prometheus. You have to setup Prometheus in your Kubernetes cluster.

And to collect kubelet metrics, you have to add the following scrape config to Prometheus:

  - job_name: 'kubernetes-nodes'
    scheme: https
      ca_file: /var/run/secrets/
    bearer_token_file: /var/run/secrets/
      - role: node
      - target_label: __address__
        replacement: kubernetes.default.svc:443
      - source_labels: [__meta_kubernetes_node_name]
        regex: (.+)
        target_label: __metrics_path__
        replacement: /api/v1/nodes/${1}/proxy/metrics
      - source_labels: [__name__]
        action: drop
        regex: kubelet_runtime_operations_duration_seconds.*

Prometheus Operator is one of the best solutions. It will automatically deploy Prometheus and configure scraping for you.

Deploy pvc-autoresizer

Deploy pvc-autoresizer as shown in the following manifest:

neco-apps/pvc-autoresizer/base at master · cybozu-go/neco-apps · GitHub

Note that you should set --prometheus-url args according to your environment.

      - name: manager
        - --prometheus-url=http://prometheus.monitoring.svc:9090

Enable auto-resizing

You can specify whether or not to enable automatic resizing for each StorageClass. If you want to enable it, add "true" annotation to the StorageClass.

kind: StorageClass
  name: my-provisioner
  annotations: "true" 
volumeBindingMode: WaitForFirstConsumer
allowVolumeExpansion: true

Prepare a PVC and a Pod

Prepare a PVC you want to auto-resize and a Pod to use it.

kind: PersistentVolumeClaim
apiVersion: v1
  name: my-pvc
  annotations: 20% 20Gi
  - ReadWriteOnce
  volumeMode: Filesystem
      storage: 30Gi
      storage: 100Gi
  storageClassName: my-provisioner must be specified. It acts as safeguard. pvc-autoresizer increases PVC's up to

In addition, there are two optional annotations. is the threshold of free space. If the free space of the target PV is smaller than the specified value, it will be resized. The default value is 10%. is the amount of increased size. The request size of the target PVC is increased by a specified percentage of the request size. The default value is 10%.


Enter into the target pod:

kubectl exec -it example-pod bash

Make sure current volume usage:

root@example-pod:/# df -h /test1
Filesystem                                         Size  Used Avail Use% Mounted on
/dev/topolvm/8ad1c617-e572-4d0d-b4e8-d66e5a572df9 1014M   34M  981M   4% /test1

Create a file that take up 90% of the volume:

fallocate -l 900M /test1/test.txt

Make sure current volume usage again:

root@example-pod:/# df -h /test1
Filesystem                                         Size  Used Avail Use% Mounted on
/dev/topolvm/8ad1c617-e572-4d0d-b4e8-d66e5a572df9 1014M  934M   81M  93% /test1

After a few minutes, the volume will be resized to 2GiB:

root@example-pod:/# df -h /test1
Filesystem                                         Size  Used Avail Use% Mounted on
/dev/topolvm/8ad1c617-e572-4d0d-b4e8-d66e5a572df9  2.0G  935M  1.1G  46% /test1

Just try it

If you just want to try pvc-autoresizer, you can use the following environment with TopoLVM deployed on kind (Kubernetes IN Docker).

Next steps

Currently, pvc-autoresize is a beta version. Please let us know if you're interested in pvc-autoresize, please join slack.

We are continually improving it and going to make it production ready!