lobsters score: 7 · tags: api, programming
This is especially true when some internal state is guarded by a mutex. In that situation there’s the danger of a function holding the mutex calling another function that acquires the mutex, causing deadlock. If you have a rule that public functions grab the mutex and internal functions can assume the mutex is held, and the class never calls its own public methods, you avoid the problem.
score 1 · canonical
Recursive mutexes exist for pretty much exactly this reason. They are the easiest way to make a non-thread-safe library thread safe: every public API call grabs the mutex and releases it on return. They’re not quite as efficient as non-recursive ones, but they’re not that expensive: typically the recursion check is done on the failure case: try to acquire the lock. If you succeeded then continue. If you failed, check if the current owner is you and increment the counter if so. On release, check if the counter is 0, drop the lock if so and decrement the counter if not. Once you move to fine-grained locking, lock acquisition tends to be further away from the public API entry points, the problems arise if you call a public API in the middle of something that is transaction shapes, and then you can get lock-order reversal.
score 1 · canonical
I’m not sure I agree with this. I tend to build public APIs in layers. The low layer public API exposes all necessary functionality. The higher layers expose convenient functionality for common use cases. In C/C++, the higher-level interfaces are often header-only and designed to be inlined in the caller. They also serve as documentation: if you have a use case that is similar to this common case, you can use the body of this wrapper API as inspiration for how you should be calling the lower layer.
The fact that the lower layer now has additional constraints on its refactoring is a feature. It’s a public API, it should be stable. If the low layer wasn’t useful for implementing the higher layer, then it also probably want useful for other consumers.
I think the real messages should be:
- Build public APIs to meet requirements and be stable.
- Understand the layering in your system and don’t have lower layers depend on higher layers (except via explicit dependency injection).
- Design your APIs to be composable.
score 1 · canonical
OP did say it’s OK to have helper functions that call only other public functions.
score 1 · canonical
It depends: for example, in Rust’s
Vecthere are a lot of pub methods that call other pub methods for a bunch of different reasons (safety, reuse, allocator API, …).
score 1 · canonical
Or you could do whatever makes the most sense at a given moment and then when the circumstances change, refactor your code so it keeps making the most sense under the new circumstances
score 1 · canonical
Which is, imo, a natural, but horrible consequence of misguided approach to testing. You shouldn’t be mocking your methods, you should mock your IO
Is mocking a
MyCustomResourceStore“mocking your IO”, or does “mock your IO” refer to some lower layer of the I/O stack? Where does I/O begin?
score 2 · canonical
That’s a good observation, that there are many different ways to cut the interfaces such that pure business is on the one side, and side-effecting syscalls are on the other. And any cut works! But I think ideally you actually want two:
- Most importantly, you cut the thinnest waist that connects your app to the external world (the trick here is architecting your application in a way that creates thin waist. Eg, you can have
MyCustomeResourceStore, our you can can sprinkle raw SQL calls all other the place). - But then, to test implementation of the thin waist itself, you want to cut at the layer of syscalls. This second cut is significantly messier, and gives relatively less bang for buck, but is still a good idea to have.
The first layer tests logical, the second layer tests physical.
score 1 · canonical
Thanks for clarifying! A couple more questions:
the trick here is architecting your application in a way that creates thin waist. Eg, you can have MyCustomeResourceStore, our you can can sprinkle raw SQL calls all other the place
Should I interpret this as “both of these are examples of a thin waist” or is one meant to be thicker than the other? My intuition is the latter—that MCRS is thinner than sprinkling SQL around, but I’m not sure if my idea of a thin waist matches yours.
But then, to test implementation of the thin waist itself, you want to cut at the layer of syscalls. This second cut is significantly messier, and gives relatively less bang for buck, but is still a good idea to have.
Ok, so assuming MCRS is the first cut, we would mock that to test everything “above it” and then we would unit test the MCRS implementation in isolation by mocking syscalls?
If so, why mock the syscalls as opposed to, say, using a real Postgres? Mocking the network syscalls to behave like Postgres seems like a herculean effort (specifically to have confidence that my mocks do what Postgres does), and I’m not sure what it buys—maybe the ability to inject faults?
score 1 · canonical
- Most importantly, you cut the thinnest waist that connects your app to the external world (the trick here is architecting your application in a way that creates thin waist. Eg, you can have
Instead of reading my comment, you should watch Gary Bernhardt’s superb talk “Boundaries”, which discusses this very question.
https://www.destroyallsoftware.com/talks/boundaries
But I’ll try to do a very brief answer to only your question. Your question sounds like it assumes a code structure that looks like this …
def my_procedure(): input_data = my_custom_resource_store.load() ... big block of logic that analyses input data and takes actions ...… and now you’re wondering whether to test the logic by mocking out MyCustomResourceStore.load(), or by mocking out the filesystem/database/network calls inside of MCRS.load().
You can sidestep that dilemma if you refactor this so the logic function does not call MCRS at all. Instead, extract a pure input-values-in, decision-values-out function. Then
- The logic can be tested with lots of unit tests because it has many paths, but no external dependencies.
- And you test MCRS.load() by an integration test, or by mocking out the lower-level IO calls.
- And the function lying above both is hopefully so trivial that it hardly needs testing – it doesn’t do IO, and it doesn’t do logic – it just threads the data from the MCRS call to the logic function, which is something the typechecker can check for you.
score 1 · canonical
Data that comes from or goes outside the program itself. This can be a file, a socket, a TTY, environment variables, command line parameters, that type of stuff. When I made my DNS library I explicitly separated packet receiving/sending from parsing/generating (you get a DNS packet, then send it to be decoded; it’s not decoded directly from the wire).
score 1 · canonical
I’ve been in the shoes of a developer adding optional function after optional function to make things “easier” for users. Some of the worst APIs grow this way. “Everything is in one function call” except that you’ve effectively reinvented having multiple functions by way of parameters.
Also, “sanctified ink” is definitely a phrase I will be using in the future.
score 3 · canonical
I see how having both
fooandbarcall a private helper improves the public API. But I immediately wondered why not have different private helpers,frobnicateandtransmogrify. (To put this another way:helper(.frobnicate)andhelper(.transmogrify)struck me as swapping the original flag boolean for a flag enum.) I guess it all depends on how much common codefrobnicateandtransmogrifywould have, though perhaps even then, the two private helpers could both call another private helper for the overlap. (I’m probably focusing too much on the details of the example, which is not fully fleshed out.)
score 1 · canonical
That was my thought too, but I assumed that it was just an example and that it shouldn’t be looked into too deeply.
score 1 · canonical