Skip to content

ENH: support new-style float_format string in to_csv #49580

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
2 of 3 tasks
joooeey opened this issue Nov 8, 2022 · 1 comment
Open
2 of 3 tasks

ENH: support new-style float_format string in to_csv #49580

joooeey opened this issue Nov 8, 2022 · 1 comment
Assignees
Labels
Enhancement IO CSV read_csv, to_csv

Comments

@joooeey
Copy link
Contributor

joooeey commented Nov 8, 2022

Feature Type

  • Adding new functionality to pandas

  • Changing existing functionality in pandas

  • Removing existing functionality in pandas

Problem Description

The float_format parameter in pd.DataFrame.to_csv doesn't support modern (i.e. since Python 2.6) format strings yet.

The documentation says about the float_format parameter:

float_format : float_formatstr, Callable, default None Format string for floating
point numbers. If a Callable is given, it takes precedence over other
numeric formatting parameters, like decimal.

The word "format string" apparently means an old-style %-format string like "%.6f". However, the Python docs tend to use that word for a modern format string like "{:.6f}".

import pandas as pd

df = pd.DataFrame([0.1, 0.2])

for float_format in ["%.6f", "{:.6f}".format, "{:.6f}"]:
    print(float_format, ":\n", df.to_csv(float_format=float_format))

Out:

%.6f :
 ,0
0,0.100000
1,0.200000

<built-in method format of str object at 0x7f62be4e01f0> :
 ,0
0,0.100000
1,0.200000

{:.6f} :
 ,0
0,{:.6f}
1,{:.6f}

Feature Description

pseudo code:

class DataFrame:
...
    def to_csv(..., float_format, ...):
        if isinstance(float_format, str):
            if "%" in float_format:
                float_format = lambda x: float_format % x
            else:
                float_format = float_format.format
        ...
        out = float_format(value)
        ...

Alternative Solutions

Workaround

df.to_csv(float_format="{.6f}".format)

Documentation change

Document that:

  • only old-style % format strings are supported.
  • one can pass in a string's format method (e.g. float_format = "{:.6f}".format) if one wants to use a modern format string.

Additional Context

If this feature is implemented and used, one could get a minor speed-up in Python 3.10 compared to using %-strings (The same speedup is accessible by using the workaround described above.):

from timeit import timeit

num = 0.1

print("%:", timeit('"%.6f" % 0.1'))
print("format:", timeit('"{:.6f}".format(0.1)'))

setup = '''
from numpy.random import rand
import pandas as pd
df = pd.DataFrame(rand(1000, 1000))
'''

print("% to_csv:", timeit(
    'df.to_csv(float_format="%.6f")', setup=setup, number=10))
print("format to_csv:", timeit(
    'df.to_csv(float_format="{:.6f}".format)', setup=setup, number=10))

Output:

%: 0.10213060600017343
format: 0.10648653099997318
% to_csv: 7.168424273999335
format to_csv: 5.367143424999995
@joooeey joooeey added Enhancement Needs Triage Issue that has not been reviewed by a pandas team member labels Nov 8, 2022
@simonjayhawkins simonjayhawkins added the IO CSV read_csv, to_csv label Feb 6, 2024
@mroeschke mroeschke removed the Needs Triage Issue that has not been reviewed by a pandas team member label Jul 16, 2024
@pedromfdiogo
Copy link

take

pedromfdiogo added a commit to pedromfdiogo/pandas that referenced this issue Jun 3, 2025
pedromfdiogo added a commit to pedromfdiogo/pandas that referenced this issue Jun 3, 2025
pedromfdiogo added a commit to pedromfdiogo/pandas that referenced this issue Jun 3, 2025
…_csv

feat(to_csv): support new-style float_format strings using str.format

Detect and process new-style format strings (e.g., "{:,.2f}") in the float_format parameter of to_csv.

- Check if float_format is a string and matches new-style pattern
- Convert it to a callable (e.g., lambda x: float_format.format(x))
- Ensure compatibility with NaN values and mixed data types
- Improves formatting output for floats when exporting to CSV

Example:
df = pd.DataFrame([1234.56789, 9876.54321])
df.to_csv(float_format="{:,.2f}")  # now outputs formatted values like 1,234.57
pedromfdiogo added a commit to pedromfdiogo/pandas that referenced this issue Jun 3, 2025
…_csv

feat(to_csv): support new-style float_format strings using str.format

Detect and process new-style format strings (e.g., "{:,.2f}") in the float_format parameter of to_csv.

- Check if float_format is a string and matches new-style pattern
- Convert it to a callable (e.g., lambda x: float_format.format(x))
- Ensure compatibility with NaN values and mixed data types
- Improves formatting output for floats when exporting to CSV

Example:
df = pd.DataFrame([1234.56789, 9876.54321])
df.to_csv(float_format="{:,.2f}")  # now outputs formatted values like 1,234.57

Co-authored-by: Pedro Santos <[email protected]>
pedromfdiogo added a commit to pedromfdiogo/pandas that referenced this issue Jun 6, 2025
…_csv

feat(to_csv): support new-style float_format strings using str.format

Detect and process new-style format strings (e.g., "{:,.2f}") in the
float_format parameter of to_csv.

- Check if float_format is a string and matches new-style pattern
- Convert it to a callable (e.g., lambda x: float_format.format(x))
- Ensure compatibility with NaN values and mixed data types
- Improves formatting output for floats when exporting to CSV

Example:
df = pd.DataFrame([1234.56789, 9876.54321])
df.to_csv(float_format="{:,.2f}")  # now outputs formatted values like
1,234.57

Co-authored-by: Pedro Santos <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Enhancement IO CSV read_csv, to_csv
Projects
None yet
Development

No branches or pull requests

4 participants