Optimize numeric jacobian for faster unit testing

[mhmukadam]

After adding support for checking cost functions Jacobians in #321.

[xphsean12]

Hello. Could I know where can this new function be found?

[luisenp]

Hi @xphsean12. Are you referring the the numeric jacobians? We haven't made this optimization yet, but note that this function is only used in our unit tests. For users the recommended way is to use our analytical jacobians if you have close form expressions for the derivatives of your costs, or if not, you can use AutodiffCostFunction.

[xphsean12]

Thanks for your advice. AutodiffCostFunction is very good, but it is a little bit slow. So I also want to specify the Jacobian to speed it up. But I am also worried about the correctness of my written one

[luisenp]

Would this be useful for you? It's an example of how we test our analytic jacobians numerically in our unit tests. You can follow a similar template to verify the correctness of yours.

We haven't yet added the function in #321 as we have been prioritizing other features, but I'll bump its priority a bit.

[xphsean12]

Would this be useful for you? It's an example of how we test our analytic jacobians numerically in our unit tests. You can follow a similar template to verify the correctness of yours.

We haven't yet added the function in #321 as we have been prioritizing other features, but I'll bump its priority a bit.

Thanks! It looks quite good, I will have a try later

[xphsean12]

Would this be useful for you? It's an example of how we test our analytic jacobians numerically in our unit tests. You can follow a similar template to verify the correctness of yours.

We haven't yet added the function in #321 as we have been prioritizing other features, but I'll bump its priority a bit.

Thanks for your support. After reading the code, I have made some modifications and made it simpler for auto-checking by utilizing AutoDiffCostFunction. Details are as belows. I modified it from my use case and skiped some irrelevant part

opt_var1, opt_var2, opt_var3, aux_var1, aux_var2, aux_var3 = prepareInputForCostFunction（）
#2. Check Jacobian
cost_weight = th.ScaleCostWeight(1.0)
test_jacobian(opt_var1, opt_var2, opt_var3, aux_var1, aux_var2, aux_var3, cost_weight)

def test_jacobian(opt_var1, opt_var2, opt_var3, aux_var1, aux_var2, aux_var3,  cost_weight):
    cost_function = exampleCost(opt_var1, opt_var2, opt_var3, aux_var1, aux_var2, aux_var3,  cost_weight)

    aux_vars = aux_var1, aux_var2, aux_var3
    optim_vars = opt_var1, opt_var2, opt_var3
    interation = aux_var1.shape[1] # or maybe aux_var1.dof()
   
    cost_function_numeric = th.AutoDiffCostFunction(optim_vars, cost_function.error_numeric, interation, aux_vars=aux_vars)
    
    
    expected_jacs, error_jac = cost_function_numeric.jacobians()
    #print(expected_jacs)
    jacobians, error_jac = cost_function.jacobians()
    error = cost_function.error()
    print("checking Jacobian...")
    assert torch.allclose(error_jac, error)
    for i in range(len(jacobians)):
        assert torch.allclose(jacobians[i], expected_jacs[i], atol=1e-8)



class exampleCost(th.CostFunction):
    def __init__(
        self,
        cost_weight: th.CostWeight,
        opt_var1: th.Vector,
        opt_var2: th.Vector,
        opt_var3: th.Vector,
        aux_var1: th.Vector,
        aux_var2: th.Vector,
        aux_var3: th.Vector,
        cost_weight: CostWeight,
        name: Optional[str] = None,
    ):
        super().__init__(cost_weight, name=name) 
        self.opt_var1 = opt_var1
        self.opt_var2 = opt_var2
        self.opt_var3 = opt_var3
        self.aux_var1 = aux_var1
        self.aux_var2 = aux_var2
        self.aux_var3 = aux_var3
        
        self.register_optim_vars(["opt_var1", "opt_var2", "opt_var3"])
        self.register_aux_vars(["aux_var1", "aux_var2", "aux_var3"])

    def error(self) -> torch.Tensor:
        # self define error 
        error = None
        return error

    def error_numeric(self, opt_var, aux_var) -> torch.Tensor:
        # self define error 
        opt_var1, opt_var2, opt_var3 = opt_var
        aux_var1, aux_var2, aux_var3 = aux_var
        # calculate error, exactly same as error()
        return error

    def jacobians(self) -> Tuple[List[torch.Tensor], torch.Tensor]:
        Jocob_List = []
        return Jocob_List, self.error()

    def dim(self) -> int:
        return self.var.dof()

    def _copy_impl(self, new_name: Optional[str] = None) -> "exampleCost":
       pass

[mhmukadam]

@xphsean12 Great! Thanks for sharing the changes to make it easier to use for auto checking. Would you be interested in opening a PR with this + some unit tests? It would be nice to add it to the library and make it more accessible for all users.

[xphsean12]

@xphsean12 Great! Thanks for sharing the changes to make it easier to use for auto checking. Would you be interested in opening a PR with this + some unit tests? It would be nice to add it to the library and make it more accessible for all users.

Thanks for your appreciation! I will try to open a PR later when I have time.

[DanielTakeshi]

Would this be useful for you? It's an example of how we test our analytic jacobians numerically in our unit tests. You can follow a similar template to verify the correctness of yours.

We haven't yet added the function in #321 as we have been prioritizing other features, but I'll bump its priority a bit.

Here's the updated link for anyone checking today: https://github.com/facebookresearch/theseus/blob/main/tests/theseus_tests/embodied/measurements/test_moving_frame_between.py

It looks like https://github.com/facebookresearch/theseus/issues/321 has been superseded by https://github.com/facebookresearch/theseus/pull/465 which was merged and that the utility is now in https://github.com/facebookresearch/theseus/blob/5db40e9c3572c18f684709eb9758d0b408902bb5/theseus/utils/utils.py#L143-L185 so the issue here might be resolved, to my understanding. Thanks for the efforts!

[luisenp]

Hi @DanielTakeshi, thanks for posting the updated info!

For completeness, I think closing the issue would be best after migrating all cost function tests to use the new utility, because a lot of our old unit tests are still based on numerical jacobian references.