Pydantic: Make field None in validator based on other field's value

Question

I'm using the pydantic BaseModel with a validator like this:

from datetime import date
from typing import List, Optional
from pydantic import BaseModel, BaseConfig, validator

class Model(BaseModel):
    class Config(BaseConfig):
        allow_population_by_alias = True
        fields = {
            "some_date": {
                "alias": "some_list"
            }
        }
    some_date: Optional[date]
    some_list: List[date]

    @validator("some_date", pre=True, always=True)
    def validate_date(cls, value):
        if len(value) < 2: # here value is some_list
            return None
        return value[0] # return the first value - let's assume it's a date string

# This reproduces the problem
m = Model(some_list=['2019-01-03'])

I would like to compute the value of some_date based on the value of some_list and make it None if a certain condition met.

My JSON never contains the field some_date, it's always populated based on some_list hence pre=True, always=True. However the default validator for some_date will run after my custom one, which will fail if validate_date returns None.

Is there a way to create such a field which is only computed by another one and still can be Optional?

@normanius my bad, forgot that allow... and fields were in an inner Config class — Dániel Nagy
– Dániel Nagy, Commented Jan 3, 2019 at 14:42
Newer versions of pydantic (starting with 0.20) handle your use case much better. — normanius
– normanius, Commented Dec 3, 2020 at 23:55

tupui · Accepted Answer · 2020-12-03 15:32:01Z

66

If you want to be able to dynamically modify a field according to another one, you can use the values argument. It holds all the previous fields, and careful: the order matters. You could do this either using a validator or a root_validator.

With a `validator`

>>> from datetime import date
>>> from typing import List, Optional
>>> from pydantic import BaseModel, validator
>>> class Model(BaseModel):
        some_list: List[date]
        some_date: Optional[date]
    
        @validator("some_date", always=True)
        def validate_date(cls, value, values):
            if len(values["some_list"]) < 2:
                return None
            return values["some_list"][0]

>>> Model(some_list=['2019-01-03', '2020-01-03', '2021-01-03'])
Model(some_list=[datetime.date(2019, 1, 3), datetime.date(2020, 1, 3), datetime.date(2021, 1, 3)],
      some_date=datetime.date(2019, 1, 3))

But as I said if you exchange the order of some_list and some_date, you will have a KeyError: 'some_list'!

With a `root_validator`

Another option would be to use a root_validator. These act on all fields:

>>> class Model(BaseModel):
        some_list: List[date]
        some_date: Optional[date]
    
        @root_validator
        def validate_date(cls, values):
            if not len(values["some_list"]) < 2:
                values["some_date"] = values["some_list"][0]
            return values

>>> Model(some_list=['2019-01-03', '2020-01-03', '2021-01-03'])
Model(some_list=[datetime.date(2019, 1, 3), datetime.date(2020, 1, 3), datetime.date(2021, 1, 3)],
      some_date=datetime.date(2019, 1, 3))

edited Dec 3, 2020 at 15:32

answered Dec 3, 2020 at 15:09

tupui

6,5463 gold badges35 silver badges55 bronze badges

Sign up to request clarification or add additional context in comments.

5 Comments

rypel Over a year ago

this should be the accepted answer, imho: "the order [of declared attributes] matters" - ran into this exact sneaky KeyError

Ramon Dias Over a year ago

does not work if you set exclude=True on some_list :(

tupui Over a year ago

@RamonDias which seems logical no? You would need to adapt the validator to handle this case, otherwise some_list is not present.

Ramon Dias Over a year ago

@tupui I thought exclude=True would only count for dumping.

nck Over a year ago

If you want to populate a filed based on another one then make sure you use always=True as in the provided answer. Otherwise if the field you want to populate is null it will not work.

normanius · Accepted Answer · 2020-12-03 23:52:01Z

9

Update: As others pointed out, this can be done now with newer versions (>=0.20). See this answer. (Side note: even the OP's code works now, but doing it without alias is even better.)

From skim reading documentation and source of pydantic, I tend to to say that pydantic's validation mechanism currently has very limited support for type-transformations (list -> date, list -> NoneType) within the validation functions.

Taking a step back, however, your approach using an alias and the flag allow_population_by_alias seems a bit overloaded. some_date is needed only as a shortcut for some_list[0] if len(some_list) >= 2 else None, but it's never set independently from some_list. If that's really the case, why not opting for the following option?

class Model(BaseModel):
    some_list: List[date] = ...

    @property 
    def some_date(self):
        return None if len(self.some_list) < 2 else self.some_list[0]

edited Dec 3, 2020 at 23:52

answered Jan 3, 2019 at 19:39

normanius

9,9698 gold badges64 silver badges97 bronze badges

2 Comments

Dániel Nagy Over a year ago

I was wondering if my approach was possible at all, but you are right, this problem shouldn’t be solved like this

mecampbellsoup Over a year ago

what's wrong with doing it this way? seems simple. why is validator better approach?

Bryan Gorges · Accepted Answer · 2020-01-21 21:12:56Z

4

You should be able to use values according to pydantic docs

you can also add any subset of the following arguments to the signature (the names must match):

values: a dict containing the name-to-value mapping of any previously-validated fields

config: the model config

field: the field being validated

**kwargs: if provided, this will include the arguments above not explicitly listed in the signature

@validator()
def set_value_to_zero(cls, v, values):
    # look up other value in values, set v accordingly.

answered Jan 21, 2020 at 21:12

Bryan Gorges

411 silver badge2 bronze badges

Comments

miksus · Accepted Answer · 2022-06-25 19:42:49Z

What about overriding the __init__?

from datetime import date
from typing import List, Optional
from pydantic import BaseModel

class Model(BaseModel):
    some_date: Optional[date]
    some_list: List[date]

    def __init__(self, *args, **kwargs):

        # Modify the arguments
        if len(kwargs['some_list']) < 2:
            kwargs['some_date'] = None
        else:
            kwargs['some_date'] = kwargs['some_list'][0]

        # Call parent's __init__
        super().__init__(**kwargs)

Model(some_list=['2019-01-03', '2022-01-01'])
# Output: Model(some_date=datetime.date(2019, 1, 3), some_list=[datetime.date(2019, 1, 3), datetime.date(2022, 1, 1)])

Note that if you modify the instance after creation, this validation is not executed.

pyInTheSky · Accepted Answer · 2024-05-07 02:44:25Z

1

In pydantic >= 2.0, the third paramater can be added to accept the validation information object ValidationInfo(config={'title': 'Foo'}, context=None, data={'bar': 'hello'}, field_name='baz') (vi in the below example)

from pydantic import BaseModel, field_validator
class Foo(BaseModel):
    bar: str
    baz: int
    @field_validator("baz", mode="before")
    def val_baz(cls, value, vi):
            print(vi)
            if vi.data["bar"] == "hello":
                    raise ValueError("oh no")
            return value

Triggers Validation Exception: f = Foo.model_validate({"bar":"hello","baz":1})
Success: f = Foo.model_validate({"bar":"hi there","baz":1})

answered May 7, 2024 at 2:44

pyInTheSky

1,4891 gold badge10 silver badges28 bronze badges

Collectives™ on Stack Overflow

Pydantic: Make field None in validator based on other field's value

5 Answers 5

With a `validator`

With a `root_validator`

5 Comments

2 Comments

Comments

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

5 Answers 5

With a validator

With a root_validator

5 Comments

2 Comments

Comments

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related

With a `validator`

With a `root_validator`