Ceph OSD處理客戶端寫(xiě)操作處理流程的示例分析

這篇文章主要介紹Ceph OSD處理客戶端寫(xiě)操作處理流程的示例分析，文中介紹的非常詳細(xì)，具有一定的參考價(jià)值，感興趣的小伙伴們一定要看完！

1、OSD從client端收到請(qǐng)求的處理流程

OSD::ms_fast_dispatch()

|__OSD::dispatch_session_waiting()

     |__OSD::dispatch_op_fast()

          |__OSD::handle_op()

               |__OSD::get_pg_or_queue_for_pg()

               |__OSD::enqueue_op()

                    |__PG::queue_op()

                         |__OSD::op_wq.queue()          將client的請(qǐng)求寫(xiě)入到隊(duì)列

主OSD處理從client端請(qǐng)求的處理流程

OSD::ShardOpWQ::_process()

|__OSD::dequeue_op()

     |__ReplicatedPG::do_request()

          |__ReplicatedPG::do_op()

               |__ReplicatedPG::execute_ctx()

                    |__ReplicatedPG::preapare_transcation()

                         |__ReplicatedPG::do_osd_ops()

                    |__ReplicatedPG::issue_repop()

                         |__ReplicateBackend::submit_transaction()

                              |__ReplicateBackend::issue_op()                    主OSD將寫(xiě)請(qǐng)求發(fā)送到從OSDs

                              |__ReplicatedPG::queue_transcations()          寫(xiě)主OSD FileJournal

                                   |__ObjectStore::queue_transactions()

                                        |__FileStore::queue_transactions()

                                             |__JournalingObjectStore::_op_journal_transactions()

                                                  |__FileJournal::submit_entry()     將日志寫(xiě)入到FileJournal的writeq隊(duì)列

                    |__ReplicatedPG::eval_repop()

主OSD的FileJournal寫(xiě)線程

FileJournal::write_thread_entry()           負(fù)責(zé)從writeq隊(duì)列中獲取待寫(xiě)入的日志

|__FileJournal::prepare_multi_write()     構(gòu)成日志

|__FileJournal::do_write()                       實(shí)際寫(xiě)日志

     |__FIleJournal::queue_completions_thru()

          |__completion_peek_front()          獲取C_JournalAhead類實(shí)例

          |__finisher->queue()

          |__finisher_cond.Signal()               啟動(dòng)FileJournal的finisher線程處理C_JournalAhead類實(shí)例

主OSD的C_JournalAhead類處理

C_JournalAhead::finish()

|__FileStore::_journaled_ahead()

     |__FileStore::queue_op()

          |__FileStore::op_wq.queue()          將數(shù)據(jù)寫(xiě)入到FileStore寫(xiě)線程隊(duì)列中

     |__ondisk_finisher.queue(ondisk)     調(diào)用ReplicatedBackend->op_commit()回調(diào)函數(shù)

主OSD的FileStore寫(xiě)數(shù)據(jù)線程類處理

FileStore::op_wq

|__FileStore::op_wq._process()

     |__FileStore::_do_op()

          |__FileStore::_do_transactions()

               |__FileStore::_do_transaction()

                    |__FileStore::_write()          落盤(pán)到FileStore中（寫(xiě)入指定的文件中）

|__FileStore::op_wq._process_finish()

     |__FileStore::_finish_op()

          |__op_finisher.queue()                    調(diào)用ReplicatedPG::op_applied()回調(diào)函數(shù)


2、從OSD處理主OSD端發(fā)送過(guò)來(lái)的請(qǐng)求

OSD::ms_fast_dispatch()

|__OSD::dispatch_session_waiting()

     |__OSD::dispatch_op_fast()

          |__OSD::handle_replica_op()

               |__OSD::get_pg_or_queue_for_pg()

               |__OSD::enqueue_op()

                    |__PG::queue_op()

                         |__OSD::op_wq.queue()          將主OSD的請(qǐng)求寫(xiě)入到從OSD處理隊(duì)列中

從OSD處理workqueue

OSD::ShardOpWQ::_process()

|__OSD::dequeue_op()

     |__ReplicatedPG::do_request()

          |__ReplicatedBackend::handle_message()

               |__ReplicatedBackend::sub_op_modify()

                    |__ReplicatedBackend::sub_op_modify_impl()

                         |__注冊(cè)commit回調(diào)函數(shù)類C_OSD_RepModifyCommit()

                         |__注冊(cè)Apply回調(diào)函數(shù)類C_OSD_RepModifyApply()

                         |__ReplicatedPG::queue_transcations()          從OSD寫(xiě)FileJournal

                                   |__ObjectStore::queue_transactions()

                                        |__FileStore::queue_transactions()

                                             |__JournalingObjectStore::_op_journal_transactions()

                                                  |__FileJournal::submit_entry()     將日志寫(xiě)入到FileJournal的writeq隊(duì)列

                    |__ReplicatedPG::eval_repop()

從OSD的FileJournal寫(xiě)線程

FileJournal::write_thread_entry()           負(fù)責(zé)從writeq隊(duì)列中獲取待寫(xiě)入的日志

|__FileJournal::prepare_multi_write()     構(gòu)成日志

|__FileJournal::do_write()                       實(shí)際寫(xiě)日志

     |__FIleJournal::queue_completions_thru()

          |__completion_peek_front()          獲取C_JournalAhead類實(shí)例

          |__finisher->queue()

          |__finisher_cond.Signal()               啟動(dòng)FileJournal的finisher線程處理C_JournalAhead類實(shí)例

從OSD的C_JournalAhead類處理

C_JournalAhead::finish()

|__FileStore::_journaled_ahead()

     |__FileStore::queue_op()

          |__FileStore::op_wq.queue()          將數(shù)據(jù)寫(xiě)入到FileStore寫(xiě)線程隊(duì)列中

     |__ondisk_finisher.queue(ondisk)     調(diào)用ReplicatedBackend->op_commit()回調(diào)函數(shù)

從OSD的日志寫(xiě)完成后的處理

C_OSD_RepModifyCommit()

|__finish()

     |__ReplicatedBackend::sub_op_modify_commit()

          |__生成MOSDRepOpReply消息                                   CEPH_OSD_FLAG_ONDISK

          |__ReplicatedPG::send_message_osd_cluster()          將從OSD回復(fù)信息發(fā)給主OSD

               |__OSD::send_message_osd_cluster()

從OSD的FileStore寫(xiě)數(shù)據(jù)線程類處理

FileStore::op_wq

|__FileStore::op_wq._process()

     |__FileStore::_do_op()

          |__FileStore::_do_transactions()

               |__FileStore::_do_transaction()

                    |__FileStore::_write()          落盤(pán)到FileStore中（寫(xiě)入指定的文件中）

|__FileStore::op_wq._process_finish()

     |__FileStore::_finish_op()

          |__op_finisher.queue()                    調(diào)用ReplicatedPG::op_applied()回調(diào)函數(shù)

從OSD的落盤(pán)完成后的處理

C_OSD_RepModifyApply()

|__finish()

     |__ReplicatedBackend::sub_op_modify_applied()

          |__生成MOSDRepOpReply消息                                   CEPH_OSD_FLAG_ACK

          |__ReplicatedPG::send_message_osd_cluster()          將從OSD回復(fù)信息發(fā)給主OSD

               |__OSD::send_message_osd_cluster()


3、主OSD處理從OSD發(fā)送Reply處理流程

從OSD處理主OSD端發(fā)送過(guò)來(lái)的請(qǐng)求

OSD::ms_fast_dispatch()

|__OSD::dispatch_session_waiting()

     |__OSD::dispatch_op_fast()

          |__OSD::handle_replica_op()

               |__OSD::get_pg_or_queue_for_pg()

               |__OSD::enqueue_op()

                    |__PG::queue_op()

                         |__OSD::op_wq.queue()          將從OSD的reply寫(xiě)入到主OSD處理隊(duì)列中

主OSD處理從OSD端發(fā)送過(guò)來(lái)的reply

OSD::ShardOpWQ::_process()

|__OSD::dequeue_op()

     |__ReplicatedPG::do_request()

          |__ReplicatedBackend::handle_message()

               |__ReplicatedBackend::sub_op_modify_reply()

                    |__ip_op.waiting_for_commit.erase()

                    |__ip_op.waiting_for_applied.erase()

                    |__對(duì)于ip_op.waiting_for_applied為空，則調(diào)用ip_op.on_applied->complete(0)，即：調(diào)用on_all_applied回調(diào)函數(shù)處理。on_all_applied回調(diào)函數(shù)在ReplicatedPG.cc::execute_ctx()函數(shù)中注冊(cè)，用來(lái)對(duì)所有副本都落盤(pán)完成后的處理。on_all_applied()函數(shù)創(chuàng)建一個(gè)MOSDOpReply消息并且設(shè)置消息的flags=CEPH_OSD_FLAG_ACK且將該消息發(fā)送給客戶端；

                    |__對(duì)于ip_op.waiting_for_commit為空，則調(diào)用ip_op.on_commit->complete(0)，即：調(diào)用on_all_commit回調(diào)函數(shù)處理。on_all_commit回調(diào)函數(shù)在ReplicatedPG.cc::execute_ctx()函數(shù)中注冊(cè)，用來(lái)對(duì)所有副本在寫(xiě)FileJournal完成后的處理。on_all_commit()函數(shù)創(chuàng)建一個(gè)MOSDOpReply消息并且設(shè)置消息的flags＝CEPH_OSD_FLAG_ACK|CEPH_OSD_FLAG_ONDISK且將該消息發(fā)送給客戶端；

以上是“Ceph OSD處理客戶端寫(xiě)操作處理流程的示例分析”這篇文章的所有內(nèi)容，感謝各位的閱讀！希望分享的內(nèi)容對(duì)大家有幫助，更多相關(guān)知識(shí)，歡迎關(guān)注億速云行業(yè)資訊頻道！