diff options
| author | Pavel Begunkov <asml.silence@gmail.com> | 2021-10-14 16:10:14 +0100 |
|---|---|---|
| committer | Jens Axboe <axboe@kernel.dk> | 2021-10-19 05:49:55 -0600 |
| commit | 258f3a7f84d11ce1d6cb1f60fe524bb37f7c1019 (patch) | |
| tree | ffae56412c1b947b3e96ed20eefbc6711e365c05 | |
| parent | f80a50a632d6f5c64729563892b0deb95d563f6d (diff) | |
io_uring: optimise rw comletion handlers
Don't override req->result in io_complete_rw_iopoll() when it's already
of the same value, we have an if just above it, so move the assignment
there. Also, add one simle unlikely() in __io_complete_rw_common().
Signed-off-by: Pavel Begunkov <asml.silence@gmail.com>
Link: https://lore.kernel.org/r/8dfeb4f84026a20172bcf82c05010abe955874ae.1634144845.git.asml.silence@gmail.com
Signed-off-by: Jens Axboe <axboe@kernel.dk>
| -rw-r--r-- | fs/io_uring.c | 4 |
1 files changed, 2 insertions, 2 deletions
diff --git a/fs/io_uring.c b/fs/io_uring.c index 86ef803db27a..18a91d268fb6 100644 --- a/fs/io_uring.c +++ b/fs/io_uring.c @@ -2640,7 +2640,7 @@ static bool __io_complete_rw_common(struct io_kiocb *req, long res) { if (req->rw.kiocb.ki_flags & IOCB_WRITE) kiocb_end_write(req); - if (res != req->result) { + if (unlikely(res != req->result)) { if ((res == -EAGAIN || res == -EOPNOTSUPP) && io_rw_should_reissue(req)) { req->flags |= REQ_F_REISSUE; @@ -2695,9 +2695,9 @@ static void io_complete_rw_iopoll(struct kiocb *kiocb, long res, long res2) req->flags |= REQ_F_REISSUE; return; } + req->result = res; } - req->result = res; /* order with io_iopoll_complete() checking ->iopoll_completed */ smp_store_release(&req->iopoll_completed, 1); } |