locust压测工具:http测试过程与定时停止
发布日期:2021-07-25 13:04:53 浏览次数:18 分类:技术文章

本文共 13173 字,大约阅读时间需要 43 分钟。

locust压测环境描述

本文环境python3.5.2locust版本0.9.0

locust示例的执行过程

上文大概描述了locust的启动了流程,本文主要是接着上文继续分析,示例代码中的http的测试的执行过程,是如何去访问远端的http接口的流程。接着就分析如何通过传入的运行时间参数来停止locust的运行。

http测试用例的执行

示例代码中访问url的代码如下:

@task(2)def index(self):    self.client.get("/")@task(1)def profile(self):    self.client.get("/profile")

其中的client就是继承自HttpLocust初始化过程中创建的,由于示例代码里面的WebsiteUser继承自HttpLocust,在初始化的过程的时候;

class HttpLocust(Locust):    """    Represents an HTTP "user" which is to be hatched and attack the system that is to be load tested.        The behaviour of this user is defined by the task_set attribute, which should point to a     :py:class:`TaskSet 
` class. This class creates a *client* attribute on instantiation which is an HTTP client with support for keeping a user session between requests. """ client = None """ Instance of HttpSession that is created upon instantiation of Locust. The client support cookies, and therefore keeps the session between HTTP requests. """ def __init__(self): super(HttpLocust, self).__init__() if self.host is None: raise LocustError("You must specify the base host. Either in the host attribute in the Locust class, or on the command line using the --host option.") self.client = HttpSession(base_url=self.host) # 初始化一个client实例

此时的client就是一个HttpSession的实例,分析该类:

class HttpSession(requests.Session):                                    # 继承自requests.Session    """    Class for performing web requests and holding (session-) cookies between requests (in order    to be able to log in and out of websites). Each request is logged so that locust can display     statistics.        This is a slightly extended version of `python-request 
`_'s :py:class:`requests.Session` class and mostly this class works exactly the same. However the methods for making requests (get, post, delete, put, head, options, patch, request) can now take a *url* argument that's only the path part of the URL, in which case the host part of the URL will be prepended with the HttpSession.base_url which is normally inherited from a Locust class' host property. Each of the methods for making requests also takes two additional optional arguments which are Locust specific and doesn't exist in python-requests. These are: :param name: (optional) An argument that can be specified to use as label in Locust's statistics instead of the URL path. This can be used to group different URL's that are requested into a single entry in Locust's statistics. :param catch_response: (optional) Boolean argument that, if set, can be used to make a request return a context manager to work as argument to a with statement. This will allow the request to be marked as a fail based on the content of the response, even if the response code is ok (2xx). The opposite also works, one can use catch_response to catch a request and then mark it as successful even if the response code was not (i.e 500 or 404). """ def __init__(self, base_url, *args, **kwargs): super(HttpSession, self).__init__(*args, **kwargs) # 调用父类的初始化方法 self.base_url = base_url # 设置Host # Check for basic authentication parsed_url = urlparse(self.base_url) # 解析url if parsed_url.username and parsed_url.password: # 检查是否配置了用户名与密码 netloc = parsed_url.hostname if parsed_url.port: netloc += ":%d" % parsed_url.port # remove username and password from the base_url self.base_url = urlunparse((parsed_url.scheme, netloc, parsed_url.path, parsed_url.params, parsed_url.query, parsed_url.fragment)) # configure requests to use basic auth self.auth = HTTPBasicAuth(parsed_url.username, parsed_url.password) def _build_url(self, path): """ prepend url with hostname unless it's already an absolute URL """ if absolute_http_url_regexp.match(path): # 检查是否是绝对路径 return path # 如果是绝对路径则直接返回 else: return "%s%s" % (self.base_url, path) # 返回完整的url def request(self, method, url, name=None, catch_response=False, **kwargs): """ Constructs and sends a :py:class:`requests.Request`. Returns :py:class:`requests.Response` object. :param method: method for the new :class:`Request` object. :param url: URL for the new :class:`Request` object. :param name: (optional) An argument that can be specified to use as label in Locust's statistics instead of the URL path. This can be used to group different URL's that are requested into a single entry in Locust's statistics. :param catch_response: (optional) Boolean argument that, if set, can be used to make a request return a context manager to work as argument to a with statement. This will allow the request to be marked as a fail based on the content of the response, even if the response code is ok (2xx). The opposite also works, one can use catch_response to catch a request and then mark it as successful even if the response code was not (i.e 500 or 404). :param params: (optional) Dictionary or bytes to be sent in the query string for the :class:`Request`. :param data: (optional) Dictionary or bytes to send in the body of the :class:`Request`. :param headers: (optional) Dictionary of HTTP Headers to send with the :class:`Request`. :param cookies: (optional) Dict or CookieJar object to send with the :class:`Request`. :param files: (optional) Dictionary of ``'filename': file-like-objects`` for multipart encoding upload. :param auth: (optional) Auth tuple or callable to enable Basic/Digest/Custom HTTP Auth. :param timeout: (optional) How long in seconds to wait for the server to send data before giving up, as a float, or a (`connect timeout, read timeout
`_) tuple. :type timeout: float or tuple :param allow_redirects: (optional) Set to True by default. :type allow_redirects: bool :param proxies: (optional) Dictionary mapping protocol to the URL of the proxy. :param stream: (optional) whether to immediately download the response content. Defaults to ``False``. :param verify: (optional) if ``True``, the SSL cert will be verified. A CA_BUNDLE path can also be provided. :param cert: (optional) if String, path to ssl client cert file (.pem). If Tuple, ('cert', 'key') pair. """ # prepend url with hostname unless it's already an absolute URL url = self._build_url(url) # 获取访问路径 # store meta data that is used when reporting the request to locust's statistics request_meta = {} # 请求头部信息 # set up pre_request hook for attaching meta data to the request object request_meta["method"] = method # 请求的方法 request_meta["start_time"] = time.time() # 请求开始的时间 response = self._send_request_safe_mode(method, url, **kwargs) # 访问请求并获取返回值 # record the consumed time request_meta["response_time"] = (time.time() - request_meta["start_time"]) * 1000 # 记录响应处理的时间 request_meta["name"] = name or (response.history and response.history[0] or response).request.path_url # get the length of the content, but if the argument stream is set to True, we take # the size from the content-length header, in order to not trigger fetching of the body if kwargs.get("stream", False): request_meta["content_size"] = int(response.headers.get("content-length") or 0) # 获取返回的文件长度 else: request_meta["content_size"] = len(response.content or b"") # 否则获取响应的整体大小 if catch_response: # 如果要包含请求信息 response.locust_request_meta = request_meta return ResponseContextManager(response) # 用ResponseContextManager包裹response else: try: response.raise_for_status() # 检查返回状态 except RequestException as e: events.request_failure.fire( request_type=request_meta["method"], name=request_meta["name"], response_time=request_meta["response_time"], exception=e, ) # 如果失败则通知所有的失败请求事件执行 else: events.request_success.fire( request_type=request_meta["method"], name=request_meta["name"], response_time=request_meta["response_time"], response_length=request_meta["content_size"], ) # 如果成功则通知所有的成功事件执行 return response def _send_request_safe_mode(self, method, url, **kwargs): """ Send an HTTP request, and catch any exception that might occur due to connection problems. Safe mode has been removed from requests 1.x. """ try: return requests.Session.request(self, method, url, **kwargs) # 调用requests的Session去请求接口 except (MissingSchema, InvalidSchema, InvalidURL): raise except RequestException as e: r = LocustResponse() r.error = e r.status_code = 0 # with this status_code, content returns None r.request = Request(method, url).prepare() return r

由该代码可知,处理的请求都是通过requests库中的Session来进行请求的,示例代码中的client都是通过requests的代码进行请求,并且还可以使用session来保持会话,从而使接口请求的时候能够带上权限检查等额外信息。

locust定时退出

由于可以在启动locust可以指定执行的时间,可以到时间退出,我们分析一下该退出函数的执行,

def spawn_run_time_limit_greenlet():        logger.info("Run time limit set to %s seconds" % options.run_time)        def timelimit_stop():            logger.info("Time limit reached. Stopping Locust.")            runners.locust_runner.quit()        gevent.spawn_later(options.run_time, timelimit_stop)

等到了run_time之后,就会执行timelimit_stop函数,而该函数就是调用了实例化的locust类实例的quit方法;

def stop(self):    # if we are currently hatching locusts we need to kill the hatching greenlet first    if self.hatching_greenlet and not self.hatching_greenlet.ready():       # 检查是否有hatch_greenlet并且没有准备好 就杀掉该协程        self.hatching_greenlet.kill(block=True)    self.locusts.kill(block=True)                                           # 杀死所有的协程组    self.state = STATE_STOPPED                                              # 更改状态为停止态    events.locust_stop_hatching.fire()                                      # 通知所有locust_stop_hatching的函数执行def quit(self):    self.stop()                                                             # 停止执行所有的greenlet    self.greenlet.kill(block=True)                                          # 主greenlet杀死

此时就是停止所有的协程执行,终止该测试用例的执行。其中events.locust_stop_hatching使用了典型的观察者设计模式:

locust_stop_hatching = EventHook()class EventHook(object):    """    Simple event class used to provide hooks for different types of events in Locust.    Here's how to use the EventHook class::        my_event = EventHook()        def on_my_event(a, b, **kw):            print "Event was fired with arguments: %s, %s" % (a, b)        my_event += on_my_event        my_event.fire(a="foo", b="bar")    If reverse is True, then the handlers will run in the reverse order    that they were inserted    """    def __init__(self):        self._handlers = []                                         # 所有待处理的handlers    def __iadd__(self, handler):        self._handlers.append(handler)                              # 添加到处理的Handler列表中        return self    def __isub__(self, handler):        self._handlers.remove(handler)                              # 移除handler        return self    def fire(self, reverse=False, **kwargs):        if reverse:            self._handlers.reverse()                                # 是否排序        for handler in self._handlers:                              # 依次遍历handler并执行            handler(**kwargs)

总结

本文主要是继续分析了locust启动之后,http的请求的处理与定时退出的功能,其中http的请求都是基于requests.Session来实现的,定时退出的功能主要还是依赖于gevent中的Group来控制所有已经运行的协程,通过停止所有运行的协程来达到关闭停止运行的目的,其中还有些许细节并没有详细说明,大家有兴趣可自行查阅相关源码。鉴于本人才疏学浅,如有疏漏请批评指正

转载地址:https://blog.csdn.net/qq_33339479/article/details/87913788 如侵犯您的版权,请留言回复原文章的地址,我们会给您删除此文章,给您带来不便请您谅解!

上一篇:locust压测工具:测试信息输出与分布式模式
下一篇:locust压测工具:启动概述

发表评论

最新留言

很好
[***.229.124.182]2024年09月20日 14时31分05秒