{"id":3572,"date":"2018-12-02T08:31:15","date_gmt":"2018-12-02T08:31:15","guid":{"rendered":"http:\/\/network.ee.tsinghua.edu.cn\/niulab\/?p=3572"},"modified":"2020-09-04T07:58:11","modified_gmt":"2020-09-04T07:58:11","slug":"deepnap-data-driven-base-station-sleeping-operations-through-deep-reinforcement-learning","status":"publish","type":"post","link":"https:\/\/network.ee.tsinghua.edu.cn\/niulab\/?p=3572","title":{"rendered":"DeepNap: Data-Driven Base Station Sleeping Operations through Deep Reinforcement Learning"},"content":{"rendered":"<p><span class=\"paper_subtitle\"><a href=\"https:\/\/network.ee.tsinghua.edu.cn\/niulab\/wp-content\/uploads\/2018\/10\/deepnap_CCN.pdf\" rel=\"https:\/\/network.ee.tsinghua.edu.cn\/niulab\/wp-content\/uploads\/2018\/10\/deepnap_CCN.pdf\">\u5218\u666f\u521dIoTJ<\/a><\/span><\/p>\n<p><span class=\"paper_subtitle\">LANGUAGE<\/span> English<\/p>\n<p><span class=\"paper_subtitle\">SOURCE<\/span> <strong><em> IEEE Internet Things J.<\/em><\/strong><\/p>\n<p><span class=\"paper_subtitle\">Published Date<\/span>: Dec. 2018<\/p>\n<p><span class=\"paper_subtitle\">ABSTRACT<\/span><\/p>\n<p>Abstract\u2014Base station sleeping is an effective way to reduce the energy consumption of mobile networks. Previous efforts to design sleeping control algorithms mainly rely on stochastic traffic models and analytical derivation. However the tractability of models often conflicts with the complexity of real-world traffic, making it difficult to apply in reality. In this paper, we propose a data-driven algorithm for dynamic sleeping control called DeepNap. This algorithm uses a Deep Q-network (DQN) to learn effective sleeping policies from high-dimensional raw observations or un-quantized systems state vectors.We propose to enhance the original DQN algorithm with action-wise experience replay and adaptive reward scaling to deal with the challenges in non-stationary traffic. We also provide a model-assisted variant of DeepNap through the Dyna framework for inferring and simulating system dynamics. Periodical traffic modeling makes it possible to capture the non-stationarity in real-world traffic and the incorporation with DQN allows for feature learning and generalization from model outputs. Experiments show that both the end-to-end and the model-assisted version of DeepNap outperform table-based Q-learning algorithm and the non-stationarity enhancements improve the stability of vanilla DQN.<\/p>\n","protected":false},"excerpt":{"rendered":"<p><a href=\"https:\/\/network.ee.tsinghua.edu.cn\/niulab\/wp-content\/uploads\/2018\/10\/deepnap_CCN.pdf\" target=\"_blank\"><img loading=\"lazy\" decoding=\"async\" class=\"alignleft size-full wp-image-117\" title=\"pdf\" src=\"https:\/\/network.ee.tsinghua.edu.cn\/niulab\/wp-content\/uploads\/2010\/08\/pdf.gif\"alt=\"\" width=\"95\" height=\"50\" \/><\/a>Jingchu Liu, Bhaskar Krishnamachari, Sheng Zhou, and Zhisheng Niu, DeepNap: Data-Driven Base Station Sleeping Operations through Deep Reinforcement Learning, <span class=\"papersource\">IEEE Internet Things J., Vol. 5, No.6, Dec.2018: 4273-4282<\/span><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_jetpack_memberships_contains_paid_content":false,"footnotes":""},"categories":[7],"tags":[99,148],"jetpack_sharing_enabled":true,"jetpack_featured_media_url":"","_links":{"self":[{"href":"https:\/\/network.ee.tsinghua.edu.cn\/niulab\/index.php?rest_route=\/wp\/v2\/posts\/3572"}],"collection":[{"href":"https:\/\/network.ee.tsinghua.edu.cn\/niulab\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/network.ee.tsinghua.edu.cn\/niulab\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/network.ee.tsinghua.edu.cn\/niulab\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/network.ee.tsinghua.edu.cn\/niulab\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=3572"}],"version-history":[{"count":2,"href":"https:\/\/network.ee.tsinghua.edu.cn\/niulab\/index.php?rest_route=\/wp\/v2\/posts\/3572\/revisions"}],"predecessor-version":[{"id":3575,"href":"https:\/\/network.ee.tsinghua.edu.cn\/niulab\/index.php?rest_route=\/wp\/v2\/posts\/3572\/revisions\/3575"}],"wp:attachment":[{"href":"https:\/\/network.ee.tsinghua.edu.cn\/niulab\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=3572"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/network.ee.tsinghua.edu.cn\/niulab\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=3572"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/network.ee.tsinghua.edu.cn\/niulab\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=3572"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}